Split Testing: What It Is and Why It Matters
Hello! This is Push.House.
Different approaches are required for promoting different verticals. Moreover, an approach popular among the audience in one region may prove ineffective in another. To determine the most effective options for advertising campaigns, advertisers employ split testing. In this article, we’ll discuss what split testing is, which elements of an advertising campaign can be tested, and how their modification affects the final Conversion Rate (CR).
Split Testing
Split testing (A/B testing) is a method of evaluating the effectiveness of replacing a specific element in a scheme while keeping others constant.
With split testing, advertisers can determine which element of the advertising campaign is most effective within the scheme. For example, identify which push notification headline attracts more users, or determine which version of an image is more clickable.
A split test involves comparing several versions of advertising campaigns, identifying the most effective elements of the scheme, and subsequently optimizing to achieve the best final statistical indicators.
For an accurate assessment of the effectiveness of changing the scheme, advertisers need to create multiple versions of identical advertising campaigns and make changes to only one single element.
Why Conduct Split Testing
Even the most insignificant detail within a campaign can cost an advertiser a partial loss of budget. To determine, exclude, or replace such variables, advertisers conduct split testing.
The goal of split testing is to find the most effective elements of the scheme and improve specific indicators of the advertising campaign’s effectiveness. The data obtained during testing can be used to optimize the advertising campaign and minimize the loss of the advertising budget.
Currently, split testing allows solving the following tasks:
- Identifying and optimizing elements that influence user clicks on the ad.
- Searching for and creating the most effective scheme.
- Improving specific statistical indicators. Well-conducted split testing and the identification of a converting scheme significantly increase not only the number of clicks and redirects but also directly affect the likelihood of a lead.
Which Elements of Scheme Can Be Tested
As mentioned earlier, approaches to promotion can vary in different regions. To determine which elements directly impact the final Conversion Rate (CR), advertisers can adjust and analyze the effectiveness of the following campaign parameters:
-
Headlines
The headline is one of the first things a user notices when looking at an ad. The click and subsequent interaction depend on how engaging and relevant it is to the user’s query.
Audience queries may differ. To determine which headline is more likely to resonate, advertisers create multiple headline variations and test each one. Click-through rate is considered a crucial metric for assessing effectiveness. We’ve written an article in our blog on how to make clickable headlines for push notifications. Feel free to follow the link, explore the material, and use the knowledge gained to create headline variations for your future ad campaign.
-
Description
The headline is not the only textual element whose effectiveness can be tested. Through split testing, advertisers can determine how well the description of a future push notification is chosen.
The text in a push notification can serve several purposes:
- Clarify the headline.
- Inform about discounts and promotions.
- Elaborate on offer characteristics and features.
- Directly call to action.
In the initial stages of split testing, it’s important to ensure that the proposed variations address each of the tasks
described above and do not overlap. This is necessary so that advertisers can later make corrective changes to the already identified approach that meets the audience’s needs.
In some cases, a higher click-through rate from the audience can be achieved by using emojis. To determine the most effective texts in such cases, it is recommended to launch several campaigns with emojis and without them.
Contact the Push.House manager to get started
-
Images
A push notification consists of two images: the main image and the icon. Depending on the chosen ad format, the relevance of each can change. For example, in-page notifications do not include a main image.
Split testing allows not only to identify the most converting approach but also to determine which creative design variations have the highest response from the audience. In case of finding an effective scheme, it is necessary to make corrective changes to the design while keeping other campaign settings intact.
During the split testing phase, advertisers can prepare several creative variations based on the offer’s specifics and the chosen approach. We discussed how to make creatives correctly in an article in our blog. Feel free to follow the link and explore the material if you missed it.
-
Landing Pages
Split testing can also be useful in determining the most effective use of landing pages. Should you direct traffic straight to a landing page or choose pre-landing pages? Testing will help determine which promotion approach provides a higher CR.
In case of replacing or excluding a landing page variation, advertisers can make direct edits to the design of the launched website. An important feature of this testing model is the alignment of the offer presented in the creative with the overall message of the landing page. Any discrepancy may lead to some traffic loss. To prevent this, it’s crucial to ensure that the creative and landing page approaches align.
By the way, you can read about the different types of landing pages in an article of the same name in our blog.
-
Badges and Button Text
This is the signature functionality of setting up a Push.House advertising campaign. Advertisers can choose from a list of suggested badges and set an icon that offers various actions with the push notification. Typically, the badge corresponds to the offer’s vertical and leads to a logical target action, such as “Download” or “Write,” for example.
Depending on the chosen button text, users will be presented with several interaction options with the ad notification. Advertisers can test various button text options to determine which one users click on more frequently. By default, the button text for push notifications is “View” and “Settings.”
-
Targeting
Targeting adjustments allow you to find the most interested audience for your offer. In order to determine which group of users is most interested in the offer, the advertiser can test the following targeting settings for the advertising campaign in Push.House:
1. GEO. Promotion region. Push.House advertisers can add or exclude specific cities and regions where users will or will not receive ad notifications.
2. Device type. Mobile or desktop. The cost per click on desktop in Push.House is 50% lower than the cost per click on mobile devices.
3. OS version. Advertisers can test the operating system versions where users will receive notifications.
4. Browser and language. Browser model that users will see or be restricted from seeing ad notifications.
5. Subscription age. Age of subscription to push notifications (from 0 to 30 days). The lesser the subscription age, the higher the traffic cost.
How to Conduct Split Testing in Push.House
Before conducting split testing, ensure that the previously launched campaign generates a minimal number of clicks. It’s important to understand that split testing is a necessary tool for improving conversion metrics, not a working model for campaign launches.
After obtaining minimal data from the initial campaign launch, identify the campaign element and metric you plan to improve. Typically, CR and CTR are used as the primary metrics.
Evaluate user behavior and predict which changes within the scheme can lead to improved results. For example, if you plan to replace the badge of the ad and thereby increase the final CR, create another creative variation where you keep all variables and ONLY change the badge icon. If you make any other changes, the results of the split test may be inaccurate, directly affecting the launch’s final performance metrics.
After completing the split test, ensure that the data obtained during the launch is sufficient to evaluate the effectiveness of the scheme. Compare the results of the previous launch with those you obtained after making changes. If the results are almost identical, it means that the element did not have a significant difference for the audience and did not affect the likelihood of a click.
If the results with the changed badge are higher than the initial launch, it can be concluded that the icon directly influenced the likelihood of a click. Keep the most effective badge within the scheme and proceed to test other elements.
Test several variations of headlines, icons, texts, and images. A high CTR will indicate which creative is most relevant to your audience. The higher the CTR, the better the creative.
It’s difficult to determine the exact duration of split testing. Much will depend on the specifics of the offer and the state of the auction at the time of the ad launch. It’s advisable to rely on the amount of data obtained from the launch. Once you have enough data for analysis, you can disable the creatives that showed the worst results. In the future, you can individualize them, ultimately achieving high efficiency metrics.
Conclusion
Split testing is a special tool in the advertiser’s arsenal. With its help, you can achieve the highest statistical values for the final campaign results. Test new approaches and maximize the benefits of promoting in Push.House.
Best of luck!
