Targeted A/B test size too small
This article discusses the Campaign Standard issue where the Target size is flagged as a warning for being too small in an A/B test. This can be attributed to the percentage of the learning population with respect to the total population.
Description description
Environment
Campaign Standard
Issue/Symptoms
You send a Subject based A/B test and find that the Target size is flagged as a warning for being too small (no documentation mentions minimum target sizes). The winning variant was manually selected when the follow-up deliveries weren’t selecting a winner.
Test population extraction: 1622 messages for each of the 2 variants (total population 32451)
Test segment size is too small for A/B testing: ‘1622’ for a recommended minimum of ‘3245’
Resolution resolution
This depends on the significance of the A/B Test to be conducted. If the total population is less than 50k, the learning population needs to be at least 10% of the total population.
recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f