A/B Testing Your Way to App Store Success

The fact that everybody wants their app to get noticed and perform well in the app stores is a no brainer.

A slightly less obvious no brainer is that it’s highly unlikely that the first combination of price point, branding, and description one tries when launching an app will perform the best in the stores. This would be an extreme case of good luck, showing extraordinary customer insight! For this reason, agencies launching an app should recognise the need to try different things to market it, to maximise interest and conversions.

Thankfully, it’s possible to handle this in a scientific and data-driven way, without resorting to mindless trial and error. The way to do it is to use A/B testing.

A/B testing involves showing some potential customers one version of your client’s app’s store page(s), and the others an alternative version. By comparing conversion rates and other metrics, you can drill down on what works – what keeps customers interested, what makes them download and purchase, and what makes them head for the “Back” button. This method of testing is extremely well established with websites, and increasingly popular with switched-on app agencies.

Here are five crucial tips to help you implement a program of A/B testing for your app(s):

1. Choose the right tools

There are a host of tools to enable you to functionally implement A/B tests. Some of the most well-known include Optimizely, Google Analytics Experiments and Kissmetrics.

The cost of using these tools can vary, as can the functionality, but essentially what you’re looking for is something that allows you to serve one app store page to some readers, and another to the remainder. You obviously need something that allows you to analyse the results in depth too.

2. Establish your baseline data

Before beginning A/B testing, it makes sense to accumulate a meaningful amount of initial data based on your standard app listing. If you don’t do this, you have nothing to truly compare against.

This doesn’t mean you must wait months before implementing A/B tests, but you should have enough data to cover usual peaks and troughs in views and sales, or you could end up with test data that doesn’t paint a realistic picture.

3. Try one thing at a time

It makes absolutely no sense to carry out an A/B test where you change multiple elements of a client’s app listing.

For example, say you change your app icon, some screenshots, and some of the descriptive text. Even if the A/B test reveals that the “new” version of the page vastly improves conversions, you will have no idea which of those changes had the impact!

a/b testing app storeTherefore, it’s crucial to try one thing at a time. This exercise is all about drilling down on which elements work. A/B testing takes planning and careful analysis of the results. At the end of it you may well end up with the third icon you tried, the last set of screenshots and the first description you ever wrote – but if you do things properly you’ll know for certain which versions work. It’s also just an important to work out which didn’t work! The importance of a good App Store Optimization tool – which can help you focus on keywords, see which competitors are using which screenshots, and view app store copy – is key.

It’s also worth mentioning at this juncture that price points are another thing you can A/B test. You will find that there is a sweet spot with pricing and conversions where the multiple will result in the most income – but it will likely take you some time to identify it.

4. Think about timings

Timing is everything with A/B tests. Specifically, meaningful results take time. It’s therefore important to maximize accuracy by planning in testing periods that are long enough to “smooth out” the usual weekly “peaks and troughs” and reduce the risk of atypical days skewing conclusions.

It’s also unwise to carry out A/B tests over “unusual” periods, such as holiday times, or during global sporting and political events when people may be otherwise diverted.

5. Consider the impact on ongoing income

If you’re carrying out A/B tests on a successful and profitable app, you may have to accept that there is some risk to revenue while they’re in progress – especially if you test out a variation that is less successful than the live one.

a/b testing app storeIn some ways, this is just part of playing the “long game” towards greater success – but as you do start to hone in on that success, you will need to consider the income implications.

One way to reduce the impact is to carry out smaller tests on more drastic changes, by sending a smaller proportion of potential customers to the experimental page, for example.

A/B testing is a great way to ensure maximum app success, and it’s value is one of many factors that illustrates the fact that an app is a living product that can be constantly improved.

Any tips we missed? Feel free to leave a comment below.

And if you’re still looking for an App Store Optimization tool, take Kumulos for a FREE spin today!