Scheduling, stopping and deploying winners
When an A/B test has a winner, you can deploy it: Personizely rolls the winning variation out to every visitor the campaign targets. For content and URL split tests this happens through the Personizely snippet and can be undone. For Shopify theme, template and price tests, deploying changes your store permanently. This page also covers scheduling, pausing and stopping campaigns.
Available on: all platforms. Deploying is available for content, URL split, theme, template and product price experiments.
Schedule, pause and stop
The status button in the campaign editor shows the campaign's status and the actions available for it.
Action | Available when | What it does |
|---|---|---|
Start | Draft, scheduled or ended | Starts the campaign now. If it was scheduled, its end date is kept |
Schedule | Draft or ended | Sets Start on and, optionally, End on dates. The campaign is Scheduled until the start date |
Edit schedule | Scheduled or running | Changes the start or end date |
Pause | Running | Stops showing the campaign and recording data. Assignments are kept |
Resume | Paused | Continues the same test. Visitors keep their variation |
Stop | Running, scheduled or paused | Ends the campaign now. For experiments other than shipping, you're then asked "Deploy campaign?" |
- Pause vs stop. Pause when you plan to continue the same test, for example during a sale that would distort results. Stop when the test is over.
- Restarting an ended campaign. Start on an ended campaign runs it again with the same variations, statistics and visitor assignments. Campaigns deployed to your Shopify store can't be restarted; clone them instead.
- Schedules run on the visitor's clock. Start and end dates are checked in the visitor's browser, so a visitor whose device clock is wrong can see the campaign slightly early or late.
- Reports keep their data. Stopping or pausing doesn't delete anything. The report shows the data collected while the campaign ran, and conversions still arrive within the Revenue attribution after it ends.
Deploy a winning variation
- Check that the test has a winner. The report shows when a variation passed your significance level. See Statistics methodology.
- Click Stop and answer Yes to "Deploy campaign?". You can also deploy later from the winning variation's menu, using Deploy.
- In Deploy campaign to website, choose the Deployed variation and review the changes it will apply.
- For template tests, choose the Themes to deploy to, and whether to Remove campaign templates after deployment.
- Click Deploy.
A variation can be deployed once the campaign has been started at least once.
What deploying does, per type
Type | What happens | Can it be undone? |
|---|---|---|
Content | The snippet shows the winning variation to every visitor who matches the campaign's pages and target, indefinitely, even after the campaign's end date | Yes: Undeploy on the deployed variation |
URL split | The snippet redirects every matching visitor to the winning variation's URL, indefinitely | Yes: Undeploy on the deployed variation |
Theme (Shopify) | Publishes the winning theme as your store's live theme | No |
Template (Shopify) | Copies the winning template over the original template in your live theme, and in any other themes you selected. Optionally deletes the campaign's extra templates | No |
Product price (Shopify) | Writes the winning variation's prices to your real Shopify products and variants | No |
Shipping, full stack | Not deployable | |
Personalizations | Not deployable. Set the control group to 0% to show a personalization to everyone it targets | |
Theme, template and price deployments change your Shopify store directly. They can't be rolled back from Personizely, and the campaign can't be started again. Clone the campaign to run a similar test.
Behavior details and limitations
- Deployed variations aren't tracked. Once deployed, no views or conversions are recorded for the campaign. The report keeps the results from the test.
- Traffic allocation becomes 100%. Deploying sets allocation to 100%, so every visitor who matches the pages and target gets the winning variation.
- The target still applies. A deployed content or URL split campaign only shows to visitors who match its target. Switch the target to All visitors before deploying if the change should apply to everyone.
- Snippet deployments depend on Personizely. A deployed content or URL split campaign keeps working as long as the Personizely snippet loads on your site. For permanent changes, consider building the winning version into your theme or site, then undeploying.
- Undeploying deactivates the campaign. Undeploy stops showing the variation, and the campaign no longer runs until you start it again.
- Starting a deployed content or URL split campaign removes the deployment. The campaign runs as a test again.
- Deploying removes the campaign from its group. If only one campaign would be left in the Campaign groups (mutually exclusive tests), the group is deleted.
- Price deployment notes. Prices are written to the original products, not the duplicates used during the test. Compare-at price rules from the winning variation are written too. See Price testing (Shopify).
- Template deployment notes. The original template's content is overwritten. Check that the winning template belongs to your live theme; the app warns you when the campaign's templates belong to a different theme. See Template testing (Shopify).
Examples
- Headline winner: a content test's variation B wins with 95% significance. You stop the test, deploy B, and later add the new headline to your theme and undeploy.
- Price increase: a +10% price variation wins on profit per visitor. Deploying writes the new prices to your Shopify products.
- Seasonal pause: a test is paused during Black Friday so promo traffic doesn't skew it, and resumed afterwards with all visitors keeping their variation.
Troubleshooting
- The Deploy option is missing → the campaign is a personalization, a shipping or full-stack test, or it hasn't been started yet.
- A deployed content variation doesn't show → the visitor doesn't match the campaign's pages or target, or the Personizely snippet isn't loading. Check with ?ply_debug=1. See Preview and QA campaigns.
- Deploy failed for a Shopify campaign → the Shopify connection may be missing permissions, or a product or theme was deleted. Reconnect the app, check that the products or theme still exist, and try again.