App Marketing • July 14, 2021

iOS 15 - native A/B testing with Product Page Optimization

iOS 15 becomes publicly available later this year (Q3/4 2021) and it’ll bring along major updates for your app store marketing.

iOS 15 update will finally bring native A/B testing through Product Page Optimization and Custom Product Pages to the App Store. It gives app marketers the ability to separate between organic and paid traffic. No more “all-size-fits all” messaging but the right message to the right audience! This can multiply your conversion rates and that’s why this news is so impactful for your app store marketing. Additionally, the iOS 15 update will come with new features for Notifications, Apple Wallet and privacy changes including Hide My Email and Private Relay which will make opt-ins more valuable. 

This article will explain how to use Product Page Optimization and Custom Product Pages both. So let’s get started!

How to use product page optimization for organic traffic?

Product page optimization in Apple App Store can be seen as the counterpart for Google Play Experiments for native A/B testing. This will be a game-changer for your app store marketing and conversion rates in the Apple App Store. It enables Apple developers to experiment, analyze, and improve their app performance via split testing (a/b/c) and gives a unique opportunity to test hypotheses in a native environment in the App Store.

With the iOS 15 update, product page optimization will allow you to test your default product page with different app creatives - icons, screenshots and app preview videos - to determine which creative variation performs best for your mobile app or game. You’ll be able to compare the performance of your current visuals of your product page (control) against three different variations that could potentially improve the conversion rate of your app.

Besides this, split tests will also give you accurate insights about the most appealing features of your app and have precise user feedback about which design and creative elements work better and in which direction you should focus your app growth strategy.

You can run the test for up to 90 days and for specific countries.

Note: Your default App Store product page will only get organic traffic, composed of search and browse traffic, if you’re using custom store pages for all your paid traffic (more about that below).

In each product page optimization treatment, all creative assets will need to go through the standard App Store review process. However, it won’t require a new version release (unless it’s the app icon). For each test, you need to decide the amount of traffic per each treatment as well.

How product page optimization is different than Google Play experiments

In the Apple App Store, you’ll be able to test only your app creatives but not app titles, subtitles and descriptions. Also the review process sets the two stores apart.

The maximum number of variants is the same for both stores.

How to use iOS 15 custom product pages in the App Store for paid traffic?

iOS 15 update gives your app store marketing finally a boost with the right message to the right audience. In other words, you can have customized messages that match the intent-driven by your ads. This can multiply your conversion rates!

You see, different channels tend to attract different target audiences - users with different demographics, interests and preferences. Thanks to customized product pages, you can start influencing your conversion rates of every segment by showing the right messaging for them. This has the potential to boost your paid UA conversion rates and return on ad spend significantly, when done right. Reach out to our Apple experts for more information!

iOS 15 Custom Product Pages
You can create up to 35 custom product pages starting this fall in the App Store. Source: Apple

iOS 15 update will enable you to drive paid user acquisition traffic up to 35 Custom Product Pages. Every page can have a different app preview video, screenshots, and promotional text. The app icon will remain the same as on your default app store product page. All the pages will have an unique Apple App Store URL that you can drive specific audiences to.

As a result, you’ll be better at converting users throughout your UA funnels. Simply because the user will have a more personalized journey from seeing the ad to downloading the app, instead of “one-size-fits-all” messaging.

How customer product pages are different than Google’s customer store listings

In the Apple App Store you’ll be able to have up to 35 custom product pages compared to only five custom store listings on Google side. Google’s option lets you target only based on location, whereas for Apple’s custom product pages you can also have several pages for one location but it's applicable for traffic coming only from outside the App Store.

Also, Google’s custom store listings give more flexibility in changing copy and creatives as Apple’s custom product pages don’t allow you to change the app icon, title and subtitle.

How to run proper native A/B testing in the App Store with product page optimization?

Once the a/b test is created, you will be able to monitor and analyze impressions, downloads, conversion rates, and performance results of the test. This will help you to identify the best performing variation for your app store product page.

Before running an a/b test in the App Store, you should keep in mind that the split test should be always localized for the corresponding country. The audience percentage should be wisely distributed between the current design and the variations.

Deep dive:
A/B testing - App Radar Academy
A/B testing - 6 steps to experiment

Correct setup of an A/B test in the App Store

The most important points to focus on during A/B testing in the Apple App Store are outlined below.

  • Audience allocation between the current version and the variations. When deciding the portion of traffic which should be allocated to each variation, keep in mind your current conversion rates, the hypotheses to be tested, the elements involved and the statistical significance (in Google Play Experiments currently 90%).
  • Duration of the split test. Leave the test running for at least 7 days to avoid seasonal spikes impacting on your results. Test should run until a significant size of your potential audience went to all of the variations tested.
  • Minimum amount of installs per variation. In order to achieve significant data about the a/b test performance results, you need to keep an eye on the amount of installs needed. It’s highly linked to the daily installs rate of the app.
  • Test localization. It is highly recommended to localize your split test in a specific country. Worldwide/global experiments could be misleading as the performance of creatives is different in every country.

Note: Common beginner mistake is to ignore the current status of the app performance when setting up a proper a/b test or split test.

Further aspects to keep in mind when a/b testing in app store:

  • Closed variations might not bring much learning. Nevertheless, all the elements of your creatives should be tested to continuously improve your store page.
  • The minimum amount of installs required for statistical significance can be high and therefore make  the test duration very long.
  • Tests won't provide user behaviour details.
  • Current statistical confidence might be unreliable.

Common mistakes of A/B testing:

  • Assuming that Android and iOS user behaviour is similar.
  • Mimicking your competitors.
  • Wrongly interpreting the test results.
  • Incorrect setup of a split test.
  • Applying results that are minor uplifts or based on early test stages.

More iOS 15 updates from WWDC21

The Apple Worldwide Developers Conference (WWDC) 2021 took place June 7 to 11.  Here’s more highlights from the event, besides above mentioned product page optimization and custom product pages.

Deep dive:
Apple App Store News 2021
Search Ads available now in China

iOS 15 comes with the new status mode “Focus”

The default mode in iOS 15 is “Not Focused,” so it will be easy to measure user adoption and any impact on engagement. 

Apple Wallet turns into Mobile Wallet

Apple Wallet is adding more features with a clear goal to replace physical wallets, including e.g. driver’s licenses, state IDs, TSA clearance IDs, amusement park passes, and airline boarding passes. E.g.  Hyatt Hotels will be rolling out the digital keys in the fall for their guests.

More privacy protection - now also for Mail

With iOS 15, the Mail app will include a Mail Privacy Protection option. This will avoid senders collecting user information such as whether the email was opened or not. Evenmore, all emails will now appear to the sender as having been opened. Users also receive a privacy report with an overview of how apps are using permissions users previously granted.

Furthermore, Apple is introducing ‘Private Relay’ as its own native VPN which will hide users’ IP addresses from advertisers. Interestingly, even Apple itself won’t be able to know what users are doing. This will restrict most geo-targeting of ads and “not having IP addresses” makes industry attempts at creating tracking workarounds much harder.

Teresa Roldan App Marketing Expert App Radar
Teresa is an app marketing expert working with App Radar tools. As a mobile growth enthusiast, she loves catching algorithm disruptions red handed and always looks as peaceful as a turtle.
Get valuable insights
right in your inbox.
Your email
Sign up