Campaign analytics: How to read and use the reports

this guide explains how personizely reports campaign performance and a/b experiments it focuses on what the main metrics mean, how they’re computed, and how to interpret statistical outputs like confidence and “winner” suggestions what we track at a high level exposure and reach for each experience original (control) each variation (personalized or experimental) goal activity attributed to the campaign exposure (e g , purchases, sign‑ups, custom events) monetary outcomes when available revenue (total attributed order value or equivalent) profit (if enabled) , with an option to include shipping in profit based views attribution model last touch attribution when a visitor completes a goal, it’s attributed to the most recent eligible campaign exposure for that visitor filters you can apply time range, device type, and specific page (for content and url split campaigns) goal (primary or any enabled goal) or a specific custom event key metrics (definitions and formulas) views total recorded exposures of an experience a single visitor can generate multiple views across pages or sessions (e g , page reloads, returning visits) views help you understand total exposure volume reach unique visitors who saw an experience at least once reach is the primary denominator for comparisons and is what cvr, revenue/visitor, and profit/visitor are based on goals hit number of attributed goal events (e g , purchases, sign ups) this counts events, not necessarily unique people revenue total revenue attributed to the experience (sum of attributed order values or equivalent) profit (if enabled) total profit attributed the ui can optionally include shipping within profit when comparing “profit per visitor ” derived metrics used in comparisons goal conversion rate (cvr) goals hit / reach × 100 revenue per visitor revenue / reach profit per visitor (profit \[+ shipping, if you enable the ui toggle]) / reach notes monetary fields reflect the total of attributed goal events within your selected filters in most a/b tests, each visitor is counted in one experience’s reach; in some personalization flows, a visitor may legitimately appear in multiple experiences over time experiments vs personalization views experiment (a/b) report one row for original (control) and one for each variation columns include views, reach, goals hit, revenue (and profit if enabled), and comparison metrics (cvr, revenue/visitor, or profit/visitor) personalization report splits results into “personalized” vs “control” segments, with totals for each statistical methodology (how significance and intervals are calculated) the default and only methodology currently used in the ui is frequentist bayesian analysis is not available yet; it will be configurable in the future what we compare conversion level comparisons (cvr) compares proportions between control and a variation revenue/visitor and profit/visitor comparisons compares average monetary outcome per visitor between control and a variation frequentist methods (used today) cvr significance two sample proportion test (z approximation) inputs are visitors reached and goals hit for control vs variation revenue/visitor and profit/visitor significance two sample test on means (normal/welch’s approximation) inputs are visitors reached and total revenue or total profit for control vs variation confidence intervals (cis) per experience, a 95% ci is shown for the chosen metric (cvr, revenue/visitor, or profit/visitor) the box plot visual shows the point estimate and ci band; overlap with control’s ci is shaded significance threshold by default, a 95% threshold is used to indicate strong evidence of a difference improvement and “winner” logic improvement (%) relative lift vs control for the selected comparison metric improvement = (variation − control) / control × 100 winner suggestion when viewing “all time” for the primary goal, the ui may suggest deploying a variation if it shows positive improvement and meets or exceeds the 95% significance threshold reading the experiment table views & reach exposure volume (raw and unique) goals hit total number of attributed goal events revenue and profit (if enabled) totals attributed to the experience compare metric (selectable) goal cvr — emphasizes goal frequency per visitor revenue per visitor — emphasizes average revenue per visitor profit per visitor — emphasizes average profit per visitor confidence the statistical significance (percentage) that the variation differs from the control confidence interval 95% range where the true metric is likely to lie for the chosen metric improvement relative lift vs control for the selected compare metric tip switch the compare metric to match your primary objective (e g , cvr for sign ups; revenue/visitor for revenue optimization) historical trends when you select a date range, charts group data into consistent time buckets (hour/day/month depending on range) in your chosen timezone and keep periods with no activity visible for continuity this helps you see when performance changed and whether effects are stable over time profit analytics (shopify only) profit metrics are available for shopify stores when profit analytics is enabled for your account here is how profit is sourced and calculated data source shopify orders imported through the shopify integration calculation for each order, profit = subtotal (pre shipping) − cost of goods sold (cogs) subtotal is taken from the shopify order subtotal (before shipping), in your store currency cogs is derived from shopify inventory unit costs for each purchased variant; if any line item lacks a unit cost or isn’t managed by shopify inventory, profit for that order is not computed shipping shipping value is captured separately in profit based reports, you can optionally include shipping in “profit per visitor” via a toggle in the ui availability profit appears only for shopify stores and only when the profit analytics feature is enabled if disabled or not applicable, profit columns and profit based comparisons will be hidden practical examples cvr example if a variation has reach = 780 and goals hit = 62, its cvr is 62 / 780 × 100 ≈ 7 95% revenue per visitor example if the same variation has attributed revenue = 14,500, then revenue/visitor = 14,500 / 780 ≈ 18 59 improvement vs control if control cvr is 6 0%, then improvement = (7 95 − 6 0) / 6 0 × 100 ≈ +32 5% best practices define a primary goal and stick to it when judging experiments decide in advance on a minimum sample size or runtime before acting on results use revenue/visitor when revenue is the primary objective; use cvr when the goal count is the focus glossary original (control) the baseline experience variation an alternative experience used in personalization or a/b testing reach unique visitors who saw an experience goal an action you track (purchase, sign up, custom event) cvr (goal conversion rate) goals hit divided by reach revenue per visitor revenue attributed divided by reach profit per visitor profit attributed divided by reach significance (confidence) how likely it is that a difference vs control is not due to random chance confidence interval a range likely to contain the true metric value (95% by default) faq why can views be much higher than reach? are views or reach more important? a single visitor can view the same campaign multiple times across pages and sessions (e g , page reloads, returning visits), which increases views reach counts each visitor only once views are helpful for understanding total exposure, but reach is the primary metric used for comparisons (cvr, revenue/visitor, profit/visitor) why does a variation with fewer views sometimes win? because comparisons are normalized per visitor (cvr, revenue/visitor, or profit/visitor), not by raw view counts can i include shipping in profit per visitor? yes if profit analytics are enabled, you can toggle shipping to be included in profit based comparisons in the ui