The Ultimate Guide to iOS Subscription Testing

Find and fix bugs so you don’t lose money due to issues in your subscription code.

David Barnard
The Ultimate Guide to iOS Subscription Testing

Testing App Store subscriptions is incredibly important, but also very hard to do well. Apple’s subscription-related documentation is... um... lacking, and Apple has never been great about providing testing resources. This guide will evolve over time as Apple makes changes to subscriptions and we figure out better ways to test. If you see anything that needs to be fixed or have anything to add, please submit a pull request!

The Basics

There are 3 distinct testing environments: Production (App Store), TestFlight (Production Sandbox), and Sandbox (Developer Builds). Each behaves slightly differently and needs to be tested independently.

1. Sandbox Testing

The developer sandbox is the first line of defense. Make sure you understand the quirks and limitations during development to save time when you move on to production testing.

2. TestFlight Testing

While not especially helpful with beta testers, you should definitely spend some time testing in the production sandbox before shipping. TestFlight behaves like the sandbox but uses production App Store accounts.

3. Production Testing

There are a few tricks to testing in production even before a release hits the App Store, but you’ll also want to keep testing live on the App Store as the app is updated.

Additional Testing Strategies

Communicating with TestFlight Beta Testers

The limitations of production sandbox subscription testing makes it tough for beta testers to adequately test subscriptions for you.

Testing Free Trials and Other Introductory Offers

You can offer a discounted price or free trial of an auto-renewable subscription so new customers can experience the value of your subscription before paying full price.

Testing Subscription Offers

Apps with auto-renewable subscriptions can offer a discounted price for a specific duration for existing or previously subscribed customers.

QA Checklist

This handy checklist will help you make sure you’ve covered your bases while testing subscriptions.

Random Tips

There are too many random, poorly documented behaviors to create a page for each. This is a collection of random things that might be helpful to know.

To test in the developer sandbox, the app has to be built as a developer build in Xcode. Developers often use this to quickly test on-device during the development process, but you can also provision devices and distribute developer builds to QA and other internal testers without having to go through TestFlight and their beta app review. To prevent apps from being distributed widely outside the App Store, Apple limits device provisioning to 100 per device type (iPhone, iPad, Apple Watch, Apple TV, and Mac) for a total of 500 devices.

Testing in the developer sandbox requires a sandbox account. Apple’s documentation on this is good and will hopefully stay up to date.

To sign in with your new sandbox account for the first time, you must attempt to make a purchase in a developer build of your app. Once you have logged into a sandbox account using this method, you can then go to the Settings app, tap “iTunes & App Store”, then scroll to the bottom to the Sandbox Account section. Here you can log in and out of different sandbox accounts for testing. If you accidentally use a sandbox account on the production App Store, that account will no longer work in the sandbox. When in doubt, create a fresh sandbox account and retest.

(The Sandbox Account section of the Settings app was introduced in iOS 12. When testing on iOS 11 or before, you’ll need to sign out of your production App Store account, then sign in with a test account when prompted within the app.)

Although they should work independently, we've noticed that if your device is logged in with a production account and a sandbox account simultaneously, sandbox purchases don't work. If possible, we recommend signing in only to your sandbox account on test devices.

Sandbox Reliability

The developer sandbox is notoriously unreliable. It’s unclear whether Apple intentionally degrades performance of the developer sandbox to mimic issues that might arise in production, or if Apple uses the developer sandbox as their own sandbox and accidentally breaks it frequently. Either way, keep in mind that even if your code is working as expected, things will sometimes go awry with the developer sandbox. When this happens, it’s a great opportunity to look for ways to handle unexpected errors.

That said, never assume an issue will magically fix itself when you launch in production. There are certain known sandbox quirks that are documented in this guide, but you should still be able to fully test purchases end to end in sandbox before shipping your app.

Subscription Duration in the Developer Sandbox

Subscription length has been significantly shortened for testing purposes. This allows developers to quickly test multiple renewals and expirations.

App Store Subscription/Trial Duration Sandbox Duration
3 days 2 minutes
1 week 3 minutes
1 month 5 minutes
2 months 10 minutes
3 months 15 minutes
6 months 30 minutes
1 year 1 hour

(The 3-day duration isn’t documented anywhere but can be found by looking at sandbox transactions.)

The subscription will automatically renew up to 6 times per account. The actual number of renewals is random. After at most 6 renewals, the subscription will automatically stop renewing. These renewals happen automatically whether the app is open or not, just like renewals on the App Store. Unlike on the App Store, there’s no option to unsubscribe or get a refund, so there’s no way to directly test those scenarios. There’s also no way to test subscription management.

Each automatic renewal is added to the payment queue. The transaction (or transactions, depending on how much time has passed) is processed the next time the app is opened. Make sure you close the app and reopen it to see the updated receipt. If you’re refreshing the receipts server-side, these additional transactions should be visible in the receipt.

Testing Procedures

Testing renewals and expiration:

  1. Subscribe to a monthly subscription.
  2. Close the app and set a 20-minute timer.
  3. After 20 minutes, launch the app again to make sure your app is still in the subscribed state.
  4. Close the app and set another 20-minute timer.
  5. After 20 more minutes (approx. 40 minutes since originally purchasing the subscription), launch the app again. It should now revert to the unsubscribed state and allow the user to resubscribe.

Test restoring purchases after expiration:

  1. Subscribe to a monthly subscription.
  2. Close the app and wait 35-40 minutes.
  3. Launch the app (it should revert to the un-subscribed state).
  4. Tap the “Restore Purchases” button.
  5. No active subscription should be found, and the user should be shown a message to that effect.

Test restoring purchases during active subscription:

One big caveat in the sandbox environment is that there is no receipt file on the device until a purchase is made. This differs from the production sandbox and production environments, where a receipt file is generated at the time of installation. To fully test restores in the sandbox, you need to add a button, gesture, or other means to revert the app to the unsubscribed state.

  1. Subscribe to a monthly subscription.
  2. Use a button/gesture to revert the app to the unsubscribed state.
  3. Tap the “Restore Purchases” button.
  4. If done before the 35-minute subscription cycle, an active subscription should be found, and the app should change to the subscribed state.

Test restoring purchases across devices:

  1. Subscribe to a monthly subscription on device A.
  2. Install the app on a device B before the subscription expires.
  3. On device B, log into the same sandbox account that was used on device A.
  4. Launch the app on device B.
  5. Tap the “Restore Purchases” button.

Test upgrades, downgrades, and crossgrades:

Because the sandbox environment doesn’t have a subscription management UI like the production App Store, you’ll need to expose buttons or other means within the app to test purchases that trigger upgrades, downgrades, and cross grades.

Important Takeaways

  • Subscriptions renew at an accelerated rate in sandbox
  • Subscriptions are automatically canceled and cannot be managed by the user
  • There's no receipt available in sandbox until a purchase is made
  • Upgrades/crossgrades don't work in sandbox

References

An app distributed via TestFlight will automatically use the production sandbox environment for purchases. Users cannot be charged in a TestFlight build, but they can view the paywall and step through the purchase process without actually spending money.

Unfortunately, the production sandbox purchase process does not mimic the App Store purchase flow, which makes it confusing for beta testers. (Apple briefly enabled an App Store-like payment flow but quickly reverted it. Here’s hoping it comes back someday!)

Subscription Duration in the Production Sandbox

Subscription length has been significantly shortened for testing purposes. This allows developers to quickly test multiple renewals and expirations. Check the section above for details and caveats.

Testing Tips

To make testing easier, it can be helpful to add a button or secret gesture to the TestFlight build of your app that switches the app between various purchase states (make sure to set a build flag that removes this in the App Store build!). And don’t forget to mention this in the TestFlight release notes when recruiting beta testers to help test the paywall.

Keep in mind that while TestFlight uses the production sandbox environment, the build that is submitted to TestFlight should be the same one that gets published to the App Store. If you have a backend with a staging and a production environment, this means that TestFlight runs on your production backend, but the purchases go to the production sandbox environment. A typical setup is:

App Version App Store Your Backend
Sandbox Sandbox Staging
TestFlight Production Sandbox Production
App Store Production Production

Testing Procedures

The testing procedure in TestFlight should be the same as in sandbox.

References

For an app that has yet to be released on the App Store, getting an early version of the app approved is a great way to test subscriptions.

Pre-launch Testing

  1. Submit a beta version of the app to App Review. Make sure to set “Version Release” to “Manually release this version” so that the app is not released on the App Store.
  2. Generate promo codes for the app. This can be done for free apps that are approved but not yet live on the App Store.
  3. Download the app from the App Store using a promo code.
  4. Subscribe.

Since this app has gone through approval, subscriptions will perform exactly as they will when the app is live on the App Store—including charging testers who subscribe and allowing testers to manage their subscription on the App Store app. You can give testers promo codes to let them test the app for free. Subscriptions paid for via promo code work exactly like paid subscriptions except that they don’t auto-renew.

One other thing to note here is that apps downloaded via promo codes before the app is live on the App Store don’t seem to have a proper receipt file in the download bundle. The receipt should be refreshed with a purchase, but this issue could be used to test the rare scenario where real users of the app somehow end up in a state where an accurate receipt is not contained in the app bundle.

New Product Propagation Times

One thing to watch out for is that the app and in-app purchases (products) might not propagate to the App Store at the same time. This seems to only impact new products, not products that have previously been released with the app. It can take more than 24 hours for a new app, app update, or new product to be available on the App Store, and the app/update might show up hours before the products can be purchased. This means that you’ll be able to download the production version of the app, but the app won’t be able to buy any products. Be patient and try again 24+ hours after the app and products are approved.

Testing Renewals and Expirations

Testing renewals and expirations is difficult in production since the subscriptions aren’t shortened. To test a monthly renewal in production, you have to wait a month. To test the expiration of an annual subscription, you have to wait a year. This is impractical, which is why Apple shortens subscription durations in the sandbox testing environments. If renewals and expirations are working correctly in those environments, they should work just fine on the App Store.

Testing Cancelations and Refunds

Since it’s impossible to test cancellations in the sandbox environments, it’s important to test them in production.

Testing cancelations:

  1. Subscribe to a monthly subscription.
  2. Make sure the app enters the subscribed state with features unlocked.
  3. Go to the App Store app and cancel the subscription.
  4. Wait a minute or two to give Apple time to add the cancelation to the payment queue.
  5. Reopen the app.

At this point, auto-renew should be disabled on the receipt. The app should remain in the subscribed state, then revert to the unsubscribed state after the current billing period ends.

Testing refunds:

  1. Subscribe to a monthly subscription.
  2. Make sure the app enters the subscribed state with features unlocked.
  3. Contact Apple and request a refund.
  4. Wait a day to give Apple time to process the refund and add it to the payment queue.
  5. Reopen the app.

At this point, the refund event should be read from the payment queue and the app should revert to the unsubscribed state.

Testing the “Restore Purchase” button:

Ideally, your app will automatically determine the subscription state and unlock or lock the app. But even if that’s working as expected, receipts sometimes need to be refreshed.

  1. Subscribe to the app.
  2. Delete the app, restart the device, and reinstall the app.
  3. Launch the app.
  4. If necessary, tap the “Restore Purchases” button.

An active subscription should be found and the app should change to the subscribed state.

Because of the limitations of testing subscriptions with TestFlight (especially the short duration of subscriptions in the production sandbox), many developers choose to automatically unlock all features for beta testers.

One thing you definitely don’t want to do is rely on beta testers in TestFlight to test the price of your app, A/B test paywalls, or anything else of the sort. Apple doesn’t allow developers to actually charge TestFlight users, and many beta testers know that, so your results would be skewed. And even if Apple did allow developers to charge TestFlight users, beta testers are typically some of your most loyal users—not at all a representative sample of people coming to the app for the first time.

That said, it can be helpful to have a few beta testers at least view the paywall and/or walk through the steps from triggering the paywall all the way to making a sandbox purchase. The tricky part is explaining this process to testers in a way that will result in helpful feedback.

Most people don’t read; at best, they skim. So locking everyone out of your app’s premium features and mentioning the limitations of the beta in the release notes isn’t a great idea. You’ll end up with confused beta testers who don’t get to test all the premium features.

One option is to unlock all features for beta testers, then mention paywall testing instructions in the release notes. That way, the people who actually read the notes can help with paywall testing and everyone else can help test the premium features.

As mentioned in the TestFlight Testing section, it can be helpful to add a button or secret gesture to the TestFlight build that will revert the app to the free state and allow savvy beta testers to turn this on and encounter the paywall as normal users would once the app is live on the App Store.

Since free trials are a type of introductory offer, most subscription app developers use this feature even if they don’t realize it.

The most important thing here is to make sure your app only presents introductory offers to users that are eligible for them.

One thing to note is that once an account has redeemed an introductory offer, that account is no longer eligible for introductory offers for any product within the same subscription group. So if you test a free trial, that offer will no longer be available to that account (there doesn’t seem to be a way to reset this other than to create a fresh sandbox account).

Ideally your app should check introductory offer eligibility and hide “Free Trial” and other offer-related messaging when the user is not eligible. However, getting this wrong is a frequent cause for rejection, so err on the side of making everyone eligible instead of everyone ineligible.

References

iOS Subscription Offers can only be redeemed in sandbox after a subscription has expired. This is related to the fact that product changes don't work in sandbox. In production, an active or lapsed subscriber will be able to redeem an iOS Subscription Offer.

References

Developer Sandbox Testing

  • Test subscription purchase
  • Test subscription renewal and expiration
  • Test restoring purchases after subscription expiration
  • Test restoring purchases during an active subscription
  • Test restoring purchases across devices

Production Sandbox Testing

  • Test subscription purchase
  • Test subscription renewal and expiration
  • Test restoring purchases after subscription expiration
  • Test restoring purchases during an active subscription
  • Test restoring purchases across devices

Production Testing

  • Pre-launch purchase test
  • Launch day purchase test
  • Cancelation test
  • Refund test
  • Restore purchase test

Using Promo Codes for Subscriptions

You can generate promo codes for subscriptions in App Store Connect, but there are several caveats to be aware of before you decide to use them.

  1. A subscription granted via promo code will not auto-renew. So if you give someone a promo code for a free month, they get that month free, then the subscription ends and they have to purchase the subscription using the normal in-app flow.
  2. If you give a promo code for a free month to an existing subscriber, using it will cancel their existing subscription and give them a free month that won’t auto-renew.
  3. If your app is a paid app, you’ll have to generate a separate promo code for downloading the app first.

Apple: Promo Codes Overview

TestFlight Invites

You don’t need to collect a user’s App Store account email address in order to invite them to TestFlight. Invites can be sent to any email address. When a user taps the unique link they receive in a TestFlight email, it will associate the invite with whatever App Store account is currently logged in to that device. Future beta emails still go to the original email address, and Apple does not add the App Store account email address to your TestFlight tester list.

Missing Receipts

When an app is downloaded from the App Store, there should always be a receipt in the app bundle. But there are times when that just doesn’t happen. One theory is that when a user backs up and restores their device using iTunes, the receipt is not transferred with the app (likely to prevent tampering with the receipt).

You’d think iOS would automatically download a fresh receipt in cases like this, but no. This is why it’s important to handle the edge case of an app bundle without a receipt and provide an easy-to-find “Restore Purchases” button to retrieve that receipt.

Trying to automatically download a fresh receipt in these cases seems like a good idea, but refreshing a receipt triggers a system-level request for a password, which many users will just cancel. So it’s best to find other ways to deal with this.

Adding New Products

Once your app is live with subscriptions, you can add new products and get them approved from App Store Connect without an app update. It's important to remember that after new products are approved, they can take 24 hours to propagate throughout the App Store and become available. Because of this, you should set up your paywall such that you can enable new product offerings remotely.

  1. Your app is live on the App Store with monthly_product_1.
  2. You create a new monthly_product_2 in App Store Connect and submit it for approval.
  3. Apple approves monthly_product_2.
  4. Wait 24 hours.
  5. Switch your app to display monthly_product_2 through some remote configuration setting.

Subscription Disclosures

Apple used to require apps to disclose all sorts of terms and conditions on any paywall screen. Those requirements have since been relaxed. You don’t even have to link to your Terms of Service and Privacy Policy on the paywall anymore (but they must be linked somewhere inside the app).

The only 3 things now required on a paywall are:

  1. Title of auto-renewing subscription
  2. Length of subscription
  3. Price of subscription (and price per unit, if appropriate)

Apple: Attracting Subscribers

Tweet: Jacob Eiting

Paywall Rejections

Over the past several years Apple has gone through several rounds of tightening App Review restrictions on paywalls. As of March 2020, here are the 3 most important things to keep in mind for subscription apps:

  1. Make sure the price is more prominent (larger font and/or more bold) than “Free Trial” or other similar language.
  2. Any mention of “Free” or “Trial” must be accompanied by the actual price, even on pages/buttons that lead to your paywall.
  3. The price displayed must be the actual price users are going to pay (e.g., you can’t say “$1 per month billed annually”—you must say “11.99 per year”).

If you're struggling with implementing or scaling in-app subscriptions check out our open source SDKs. They handle the pain points of in-app subscriptions, so you can get back to building your app.

Share this post