Theme testing (Shopify)
6 min
A theme test shows a different, unpublished Shopify theme to a share of your visitors and compares it with your live theme. Use it to test a redesign, a new theme or a heavily edited copy of your current theme before publishing it. You can also run it as a personalization, showing a theme to a specific audience.
Available on: Shopify. Plan: theme campaigns must be included in your plan. Available as: A/B test · Personalization
Test or personalize. You can run a theme campaign as an A/B test, where visitors are split at random and you measure which version wins, or as a personalization, where each variation is shown to a specific audience, with an optional control group to measure the lift. The setup is the same; Personalizations covers what's different.
How it works
- Your published theme is the control. Each variation is another theme from your Shopify Themes library.
- Variation visitors are sent into a theme preview. On their first page view, the snippet reloads the current page once, telling Shopify to show the variation's theme. Shopify then keeps the visitor in that theme for the rest of their preview session, so later pages load directly in the variation theme without another redirect.
- Shopify's preview bar is hidden. Personizely hides Shopify's theme preview bar and removes the preview parameters it doesn't need from the address bar.
- The view is recorded once the variation theme loads. This requires the Personizely app embed to be enabled in the variation theme too.
- Checkout is unchanged. Visitors check out normally. Orders are attributed server-side from Shopify order data.
- When the campaign stops, visitors are taken out of the preview. A visitor still in a variation theme's preview is returned to your published theme on their next page view.
Set up a theme test
- In Shopify, duplicate your live theme (or add a new one) and make the changes you want to test. Don't publish it.
- In the theme editor of that theme, go to App embeds and make sure Personizely is enabled.
- In Personizely, go to Campaigns → New campaign → Theme, and choose Experiment or Personalization.
- For each variation, Select theme. Only unpublished themes that Shopify can preview are listed; your published theme is always the control.
- Use Preview on a variation to check it, then set the target, goals and traffic split, and start the campaign.
Option | What it does | Values / default |
|---|---|---|
Theme (per variation) | The unpublished theme shown to the variation's visitors | Any previewable, unpublished theme |
Target | Who can enter the campaign | All visitors, or a specific target |
Traffic allocation / control group | Who enters the test, and the share kept on your live theme | 100% / 50% for experiments |
Goals | What counts as a conversion | At least one goal; Purchase with revenue per visitor is typical |
A theme test runs on every page of your store. It has no page setting.
Behavior details and limitations
- The app embed must be on in every variation theme. If Personizely isn't enabled in a variation theme, the campaign editor shows a warning on that variation. Visitors would still be redirected into the theme, but no views would be recorded for it.
- One redirect per preview session. Variation visitors are redirected once, on their first page view; later pages load straight in the variation theme. Theme tests don't offer Reload the page for the control group.
- The theme preview is Shopify's. Shopify decides how long a preview session lasts. When it ends, a returning variation visitor is redirected into the preview again on their next page view.
- Shared store data. Products, cart, customer accounts, apps and checkout are the same in every theme. Only the theme's own code, settings and content differ. App blocks and embeds must be set up in each variation theme.
- Theme changes during the test. Edits you make to the variation theme are live for its visitors immediately. Edits to the published theme reach the control group immediately.
- Deleted themes. If a variation's theme is deleted in Shopify, the variation shows Removed theme. Stop the campaign or select another theme.
- Cookies. Personizely sets a first-party cookie to remember the visitor's theme preview. See Cookies and consent.
- Crawlers see your live theme. Bots don't run the snippet, so they're never redirected into a preview.
- Deploying publishes the theme. Deploying the winning variation publishes its theme as your store's live theme. This is permanent from Personizely's side (you can republish your old theme in Shopify), and the campaign can't be restarted; clone it to run a similar test. Personalizations can't be deployed.
Running it as a personalization
- One target per variation, first match wins. Each variation has its own target, and a visitor gets the first variation in the list whose target they match. Put narrow audiences above broad ones. A visitor who matches no variation sees your published theme.
- Control group. 25% of matched visitors by default stay on your published theme and are counted as control.
- Live or Permanent. Moving a visitor into or out of a theme takes a page reload. Permanent keeps each visitor on the first theme they get, so they aren't switched between themes mid-visit when their audience changes.
- No deploy. Personalizations can't be deployed, so the report has no Deploy. A personalization keeps running until you stop it, and you can edit its variations and targets as needed. To stop measuring and show it to every matching visitor, set the control group to 0%.
- Example audience: a different theme for wholesale customers, targeted with the Customer filter on customers tagged wholesale.
Examples
- Redesign before launch: a duplicate of your live theme with a new homepage and product page layout, tested 50/50 for two weeks on Purchase with revenue per visitor.
- Theme migration: your current theme against a new Shopify theme you've set up with the same apps, to check the new one doesn't lose conversion.
- Wholesale storefront: a theme personalization that shows a B2B-oriented theme to customers tagged wholesale.
Troubleshooting
- The variation has no views → the Personizely app embed is disabled in that theme.
- Visitors stay on a theme after the test ended → their preview is cleared on their next page view on your store. If it persists, check that the Personizely app embed is enabled in that theme.