Shipping rate testing (Shopify)
A shipping test shows different shipping rates at checkout to different groups of visitors, such as a lower free-shipping threshold, a cheaper standard rate or a new express option. You measure the effect on conversion, AOV and profit. You can run it as an A/B test or as a personalization for a specific audience. Your delivery profiles are restored when the campaign ends.
Available on: Shopify Plus (including Plus trial), Shopify Advanced and development stores (the campaign card shows a Shopify Advanced badge). Plan: shipping campaigns must be included in your plan. Only one shipping campaign can be active per store at a time. Available as: A/B test · Personalization
Test or personalize. You can run a shipping campaign as an A/B test, where visitors are split at random and you measure which version wins, or as a personalization, where each variation is shown to a specific audience, with an optional control group to measure the lift. The setup is the same; Personalizations covers what's different.
How it works
- You choose the rates to test. You pick zones from your Shopify delivery profiles, then the fixed-rate shipping methods in those zones that the test replaces. Each variation defines its own version of those methods, and can add new ones.
- On start, Personizely takes over those methods. It creates a carrier service in your store (called "Personizely shipping carrier"), adds it to the zones you selected, and deactivates your original methods in those zones. Zones and methods you didn't select are untouched.
- Visitors are assigned on the storefront. The snippet assigns the visitor to the control or a variation and writes it to the cart's line items (see What Personizely changes).
- Checkout asks Personizely for rates. When the visitor reaches the shipping step, Shopify requests rates for each delivery profile in the cart. Variation visitors get their variation's rates. Control visitors get your original rates, with the same names, prices and conditions. Rates whose conditions the cart doesn't meet are left out.
- The exposure is counted at the shipping step. A visitor counts as exposed when rates are returned for them at checkout, not when they view a page.
- On stop, everything is restored. Your original methods are reactivated and the carrier service is removed.
Requirements
- Shopify Plus (including Plus trial), Advanced or a development store. Shopify makes app-provided (carrier-calculated) shipping rates available on these plans.
- Extra permissions: when you create your first shipping campaign, the app asks you to approve access to read and write shipping settings and to read locations.
- At least one goal. Purchase with revenue or profit per visitor is typical.
Set up a shipping test
- Go to Campaigns → New campaign → Shipping, and choose Experiment or Personalization.
- Select the delivery profile zones to include. Personizely loads your current profiles from Shopify.
- Select the fixed-rate methods to test in each zone. Carrier-calculated rates are shown for reference only and can't be replaced.
- For each variation, edit the methods with Edit or add a shipping rate, or click Add shipping rate to offer a new one.
- Optional: click Edit in builder to change shipping text on your storefront for the variation, such as a free-shipping banner. This works like a content campaign.
- Set the target, goals and traffic split, and start the campaign.
Shipping rate fields
Field | What it does | Limits / values |
|---|---|---|
Name | The rate name shown at checkout | Required, up to 128 characters |
Description | Shown under the name at checkout | Optional, up to 128 characters |
Price | The rate's price, in your store currency | 0 shows as Free |
Condition Type | When the rate is offered | No conditions, Based on order price, Based on cart weight |
Min / Max | The condition's range | Order price in store currency; weight in your store's unit (kg, g, lb or oz). Empty = no limit. |
Delivery days (min) / (max) | Estimated delivery time | Optional |
What Personizely changes in your store
Item | When | Details |
|---|---|---|
Carrier service "Personizely shipping carrier" | While a shipping campaign is active | Removed on stop |
Carrier methods in the selected zones | While active | Shopify may briefly add the carrier to other zones; Personizely removes it from zones outside the campaign |
Your original methods in the selected zones | While active | Deactivated, then reactivated on stop |
Line item properties __ply_campaign_id, __ply_variation_id (empty for control), __ply_visitor_id | On every cart change by an assigned visitor | Properties starting with an underscore are normally hidden from shoppers by Shopify themes |
Shopify's admin activity log shows entries such as "rate was deleted … by Personizely" when the campaign starts and stops. These come from the carrier changes above; your original methods are deactivated, not deleted.
Behavior details and limitations
- One active shipping campaign per store. You can't start a second one until the first is stopped.
- Exposure means reaching the shipping step. Reach, conversion rate and per-visitor metrics are measured among visitors who got shipping rates at checkout. If you also edit storefront pages with Edit in builder, visitors who view those pages are counted as exposed too.
- Assignment needs the cart. Rates are chosen from the line item properties the snippet adds. A checkout created without them, for example from a cart built by another app's API, gets the control rates.
- Changing delivery profiles during the test. If a zone in the campaign is deleted, or every method selected in a zone is deleted, the campaign is paused automatically, which restores your original setup. If only some selected methods are deleted, they're dropped and the campaign keeps running.
- Moving products between profiles. Personizely records which products belong to which profile when the campaign starts. If you move products between profiles or add products to a custom profile while it's running, pause and resume the campaign to refresh this.
- Editing your methods during the test. Your original methods in the selected zones are inactive while the test runs. Avoid editing them in Shopify until the test ends; to change what a variation offers, edit the campaign.
- Carrier-calculated rates. Rates from carriers or other apps in the selected zones aren't replaced, and keep showing alongside the test rates.
- No preview, no deploy. Shipping campaigns can't be previewed on the storefront (test them with a real checkout) and can't be deployed. To roll out a winner, update your delivery profiles in Shopify, then stop the campaign.
- Cart progress widgets. A Cart progress element with Source: Campaign pointing at this shipping test takes its free-shipping threshold from the visitor's variation, so the bar matches the rates they'll see.
Running it as a personalization
- One target per variation, first match wins. Each variation has its own target, and a visitor gets the first variation in the list whose target they match. Put narrow audiences above broad ones. A visitor who matches no variation sees your original rates at checkout.
- Control group. 25% of matched visitors by default get your original rates at checkout and are counted as control.
- One shipping campaign at a time. The one-active-campaign limit covers experiments and personalizations together, so a shipping personalization can't run while a shipping test does.
- Live or Permanent. Use Permanent when the audience depends on something that changes during the visit, such as cart value, so the rates a shopper gets at checkout match the variation they were first given.
- No deploy. Personalizations can't be deployed, so the report has no Deploy. A personalization keeps running until you stop it, and you can edit its variations and targets as needed. To stop measuring and show it to every matching visitor, set the control group to 0%.
- Example audience: free shipping for returning customers, using the Order history filter (placed at least one order).
Examples
- Free-shipping threshold: control free shipping over $75; variation free shipping over $50. Primary metric: profit per visitor (with shipping counted), to see whether higher conversion pays for the shipping cost.
- Flat rate vs free: control $5.95 standard shipping; variation free standard shipping. Measure conversion rate and revenue per visitor.
- Express upsell: a variation that adds "Express (1–2 days)" for $12.95 next to standard shipping, to test demand for faster delivery.
Troubleshooting
- Shoppers in the variation see control rates → the checkout was created without Personizely's line item properties. Check the cart in /cart.js, and test the path the shopper used (cart page, cart drawer or another app's cart).
- No rates at all at checkout → the cart's weight or total is outside every rate's condition range. Add a fallback rate with no conditions to each variation.
- The campaign paused by itself → a zone or all selected methods in a zone were deleted in Shopify.