Configuring Experiments
Before setting up an experiment, make sure you've created the products and offerings that you want to test and added any new products to the appropriate entitlements in your project (more on this below). You should also test the offerings you've chosen on any platform your app supports.
Setting up a new experiment
Once enabled, you can access the Experiments tab under Product Setup.
Select + New to create a new experiment.
Creating a new experiment
Then, enter the following details:
- Experiment name
- Enrollment criteria (optional)
- Control variant: First offering
- Treatment variant: Second offering
Setting up an offering for your treatment
If you've not setup multiple offerings before, you'll be prompted to do so now, since you'll need at least 2 available offerings to run an experiment.
The treatment offering represents the hypothesis you're looking to test with your experiment (e.g. higher or lower priced products, products with trials, etc).
For App Store apps, we recommend setting up new products to test as a new Subscription Group so that customers who are offered those products through Experiments will see only that same set of products to select from their subscription settings.
Enrollment criteria
You may choose to setup custom enrollment criteria for your experiment to target a specific segment of your customers. To do this, click on the caret to expand the "Enrollment Criteria" section.
You may modify any of the following enrollment criteria:
| Dimension | Description |
|---|---|
| Apps | Which of your RevenueCat apps the experiment will be made available to. Set to All apps by default. |
| Countries | Which countries are eligible to have their new customers enrolled in the experiment. Set to All countries by default. |
| New customers to enroll | The percent of new customers meeting the above criteria that will be enrolled in your experiment. Set to 100% by default. |
Once done, select CREATE EXPERIMENT and view your new experiment.
Starting an experiment
When viewing a new experiment, you can start, edit, or delete the experiment.
- Start: Starts the experiment. Customer enrollment and data collection begins immediately, but results will take up to 24 hours to begin populating.
- Edit: Change the name, enrollment criteria, or offerings in an experiment before it's been started. After it's been started, only the percent of new customers to enroll can be edited.
- Delete: Deletes the experiment.
Sandbox
Test users will be placed into the experiment offering variants, but sandbox purchases won't be applied to your experiment.
If you want to test your paywall to make sure it can handle displaying the offerings in your experiment, you can use the Offering Override feature to choose a specific offering to display to a user.
FAQs
| Question | Answer |
|---|---|
| Can I edit the offerings in a started experiment? | Editing an offering for an active experiment would make the results unusable. Be sure to check before starting your experiment that your chosen offerings render correctly in your app(s). If you need to make a change to your offerings, stop the experiment and create a new one with the updated offerings. |
| Can I edit the enrollment criteria of a started experiment? | Before an experiment has been started, all aspects of enrollment criteria can be edited. However, once an experiment has been started, only new customers to enroll can be edited; since editing the apps or countries that an experiment is exposed to would alter the nature of the test. |
| Can I restart an experiment after it's been stopped? | After you choose to stop an experiment, new customers will no longer be enrolled in it, and it cannot be restarted. If you want to continue a test, create a new experiment and choose the same offerings as the stopped experiment. (NOTE: Results for stopped experiments will continue to refresh for 28 days after the experiment has ended) |
Updated 17 days ago