ASO Advanced

A/B Testing Your App Store Listing

What is App A/B Testing?

App A/B testing (Split-testing) is a method of hypothesis checking in which two options are compared in order to determine which one performs better.

An audience is distributed equally among two variations of a store page element (icon, screenshots, title, short description, etc.). Each audience member acts like an average user would. Thus, you identify which version has the best conversion rate.

App A/B testing
App A/B testing
Source: Splitmetrics

By running app A/B experiments and implementing results, app developers:

  • Understand behavioral patterns of different user segments.
  • Improve an app conversion rate (an average app conversion rate is 26%).
  • Boost organic traffic.

Сrafting an App A/B testing strategy

It’s important to figure out each subsequent move before proceeding with A/B testing of an app. Here are essential elements of any A/B testing strategy:

1. Analysis and Brainstorming Before A/B Testing

There’s no use running an A/B test without a solid hypothesis which should be based on thorough research. It’s necessary to set goals and formulate what kind of experiment you want to conduct. 

The question you’re asking should be specific and leave little room for interpretation or interfering factors. For example, the question, “are my app screenshots increasing my conversion rates?” is too general. Instead you want to ask, “do screenshots that are primarily blue convert better than ones that are primarily white?”. 

The more questions you come up with, the more A/B tests you can run. 

2. Creating variations to A/B Test

On this step, you should prepare variations of your app store listing in accordance with the hypothesis you plan to test. This could be using different app icons or app screenshots. Using the example above, you would then make one set of screenshots where the background is blue and one set of screenshots where the background is white. 

Another example of design variation could  be saliency, meaning having dynamic and eye-catching elements in your app store listing visuals, vs. generic. 

Test if salient, eye-catching, app screenshots convert better than generic ones. This is an example of an app a/b test
Test if salient, eye-catching, app screenshots convert better than generic ones.
Source: Splitmetrics

3. Running the A/B Test experiment

After you’ve created your 2 variations, you’re ready to start running the experiment. A true A/B test randomly distributes the 2 variations to your audience. However, there’s one more step to get even more specific and effective results. Identify not only which variation converts better but also which variation brings higher-quality users to your app. This way you ensure that you’re optimizing your app store listing to appeal to your target audience. 

4. Evaluating Your App A/B Test

Here is where you discover the winning variation. Sometimes your sample size may be too small to truly have a statistically significant result. However, large differences between the conversion rates of your variations will signify that one does convert better than the other. This is the variation you should implement in your app store listing. 

It’s also possible that you get no result, meaning the differences in your variations don’t really matter. Don’t give up here! You have to dig deeper at this point. If the color of the screenshots doesn’t change your target audience's behavior, maybe language does or the features that are highlighted do. 

Analyze the results of your app A/B test to identify the winning variation
Analyze the results of your app A/B test to identify the winning variation 
Source: Splitmetrics

5. Implementing collected data

If your experiment identified a clear winning variation you can either update your app store listing straight away or use these data in succeeding tests. Remember, the more you test, the better optimized your app store listing will be. 

6. Running follow-up experiments

Any app store is an ever-changing system. That’s why it’s vital to have your finger on the pulse and keep testing even when results of initial tests helped attain your aims. There are always perspectives for growth.

App A/B testing platforms

App A/B testing may present considerable difficulties though. Thankfully, there are a few genuinely efficient tools out there.

SplitMetrics is the leading A/B testing platform. You can test every single product page element both for App Store and Google Play apps there. It also allows to run pre-launch experiments, which give you a chance of preparing for a new app release in the best possible way.

If you work with Android apps only, you can opt for Google Play Experiments then. Facebook ads is another way of testing different screenshots and icons design concepts.

The importance of app A/B testing

Many publishers underestimate store page elements impact on conversion.

The truth is that optimizing your app’s page, you don’t only maximize its performance on paid traffic but also have a positive longtime effect upon organic users without extra cost. Let alone useful analytical insights which should be applied in further perfecting of your ASO strategy.

Leave the Hard Work to Us

The App Radar team will manage your apps and grow them through organic and paid user acquisition methods. We work with you to build a strategy that suits not only your needs but also your budget.

Contact Us Today
Genie is the Marketing Manager at App Radar. She drives exciting in-house campaigns and collaborations to get the word out about app marketing and growth strategies.