Targeted A/B test send has become stuck at 'Sending (Reply) and will not progress

Description

Issue statement - We sent out our first Subject-based A/B test and found the Target size flagged as a warning too small (no documentation mentions minimum target sizes). The winning variant was manually selected when the follow-up deliveries weren’t picking a winner.


The initial balance of customers was sent, but the two follow-ups deliveries are not stuck in ‘Sending (reply)’ status and will not send, with 43% of our sends stuck.


We need to get an answer or solution to the current send issue and send out the remaining emails (in a controlled release).


Resolution

The customer has a recurring AB Test delivery fed to the population from a workflow. The learning population is set as 5% per variant, and there are two variants with a learning period of 45 min. The 57% population is the initial population on which the AB test has run; after that winner has been pushed. The aggregation period of the delivery is 24 hours; that’s why the population is still being added to the delivery even after the AB test has been completed. The product has a limitation that the subsequent population doesn’t get scheduled when the winner has been pushed. This is reported as a low priority bug CAMP-47125

Workaround - Set the aggregation to none. The A/B test would not face the above issue.

On this page