Follow

Introduction

View a list of all currently active A/B Tests in View A/B Tasks.

As soon as the A/B Test Samples are deployed, a separate campaign is created for each sample.  While you can view the statistics of the samples as they collect data (along with all the other campaigns in your realm) by going to REPORTS > VIEW CAMPAIGNS, the A/B Tests in Progress page will provide a more concise view of the samples that are still collecting data and waiting to complete.

Accessing A/B Tests in Progress

To view all active A/B Test campaigns, go to TASKS > VIEW A/B TASKS.  This will display the A/B Tests in Progress page.

The A/B Tests in Progress page displays a list of all A/B Sample Tests currently active.  The samples display in this page for the first 144 hours that the campaigns are active.

Once the A/B Test is complete and the winning Sample has been deployed to the Remainder, the test will no longer display in the A/B Tests in Progress page.  To see the results of the Sample and Remainder campaigns, go to REPORTS > VIEW CAMPAIGNS and click to view the Campaign Details page for each Sample and Remainder campaign.

The A/B Tests in Progress page is split into to areas: event chart and list of active tests. 

Event Chart

The Event Chart at the top of the page displays the event rate per hour/day for each Sample in the test.  The title of the chart reflects the type of data being measured and displayed for the test:

  • Unique Open Rate: Number of Unique Opens divided by Pieces Delivered.
  • Unique Click Rate: Number of Unique Clicks divided by Pieces Delivered.
  • Click-to-Open Rate: Number of Unique Clicks divided by total number of Unique Opens.
  • Responder Rate: Number of Unique Clicks and Opens divided by Pieces Delivered.

The endpoint to stop collecting data for an A/B test is indicated in two ways:

  • Red vertical line: End Date and Hour for data collection appears for all samples  
  • Orange horizontal line: Event threshold for collecting data only appears for samples that have specific threshold set

The rates for each sample appear as colored points and lines on the chart.  Directly below the chart is a Legend bar indicating the color and current event rate associated with each Sample.  The Remainder size is indicated on the far right of this bar.

The results of the most recently run A/B Test appear in the chart when the page first displays.  To view the results of a different test, click once to select it from the list of active tests at the bottom of the page.

List of Active Tests

The list of active test Samples includes several details to help distinguish each test Sample. You may sort the data in any of the columns by clicking the column header.  The details displayed for each Definition include:

Test Name

The Name of the A/B Test Definition used to generate test campaigns.

Testing

The type of test defined: Subject Line, Template and/or From Address.

Measuring

The event statistic used to determine the winner.

Samples

The number of samples defined for the test, from 2 to 10.

Started

The date/time the Sample campaign was deployed.

Run For

The number of hours to collect test data.

End Conditions

The conditions that must be met to complete the test depending on how End is determined:

  • Test Period: Specific end date/time
  • Threshold: Rate type and percentage

Progress

A miniature chart indicating the progress of the test samples:

  • Test Period: Progress %
  • Threshold: Miniature bar chart

Status

The current status of the test:

  • In progress: Waiting to meet End Condition
  • Test complete: End Condition has been met, Remainder has been sent
  • Test Complete, select winner: End Condition has been met, Remainder is waiting to be sent
  • Test failed: Test not deployed due to failure
  • Test not started yet: Workflow is waiting approval
 
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk