This is an old revision of the document!
How does split testing work?
What is an A-B test / split test?
It is often difficult to predict how readers will react to a campaign. This is why the most important principle in e-mail marketing is: test, test, test. The automatic A/B split tests in promio.connect make it easy to test a campaign. Set up two or more e-mails (with differences in – for example – subject line, sender or content). Each of these e-mails is sent to a subset of the target group. The winning e-mail (with the highest open, click or conversion rate) is automatically sent to the rest of the target group with a configurable time offset.
A split-run test allows you to compare different designs or subject lines in a test mailing prior to the main delivery. In the main delivery, the best performing mailing is then automatically sent to the rest of the recipients. When creating the campaign, select the type A/B Test Campaign.
This box – which only becomes visible after setting up the creatives – contains all necessary settings for split testing.
- Target group: Unlike standard campaigns, split testing requires the target group to be selected at the campaign level. The target group contains the addresses for both test run and main delivery.
- Test Part of target group: The percentage of the target group that will be used for the test run.
- Duration test period: Defines the period of time between test run and main delivery in hours. In this time the winner gets identified.
- Winner Parameter: The message with the highest value in this category is declared A/B test winner. It can be determined on the basis of conversions, clicks or opens.
The delivery date that has been inserted under Actions > Send campaign > Delivery date refers to the start of the test part. The main delivery starts after finishing the test period.
Example:
- Duration test period: 5 h
- Campaign start: 10 o’clock
The test run starts at 10 o’clock, the main delivery follows at 15 o’clock.
Please keep in mind that
- coupon codes cannot be used in an A/B test campaign.
- there is no option in A/B test campaigns to refresh the target group. This means that you should finalize your split tests as late as possible - so you do not address those who signed out.
