---
title: How to read and use the reports
slug: how-to-read-and-use-the-reports
docTags: 
createdAt: 2025-11-04T16:51:45.289Z
---

This guide explains how Personizely reports campaign performance and A/B experiments. It focuses on what the main metrics mean, how they’re computed, and how to interpret statistical outputs like confidence and “winner” suggestions.

### What we track at a high level

- Exposure and reach for each experience:
  - Original (Control)
  - Each Variation (personalized or experimental)
- Goal activity attributed to the campaign exposure (e.g., purchases, sign‑ups, custom events)
- Monetary outcomes when available:
  - **Revenue** (total attributed order value or equivalent)
  - **Profit (if enabled)**, with an option to include shipping in profit-based views

Attribution model

- Last-touch attribution: when a visitor completes a goal, it’s attributed to the most recent eligible campaign exposure for that visitor.

Results can be viewed by **Goal** (primary or any enabled goal).

### Key metrics (definitions and formulas)

- **Views:** total recorded exposures of an experience. A single visitor can generate multiple views across pages or sessions (e.g., page reloads, returning visits). Views help you understand total exposure volume.
- **Reach:** unique visitors who saw an experience at least once. Reach is the primary denominator for comparisons and is what CVR, Revenue/visitor, and Profit/visitor are based on.
- **Goals hit:** number of attributed goal events (e.g., purchases, sign-ups). This counts events, not necessarily unique people.
- **Revenue:** total revenue attributed to the experience (sum of attributed order values or equivalent).
- **Profit (if enabled):** total profit attributed. The UI can optionally include shipping within profit when comparing “profit per visitor.”

Derived metrics used in comparisons

- Goal Conversion Rate (CVR): Goals hit / Reach × 100
- Revenue per visitor: Revenue / Reach
- Profit per visitor: (Profit \[+ Shipping, if you enable the UI toggle]) / Reach

Notes

- Monetary fields reflect the total of attributed goal events within your selected filters.
- In most A/B tests, each visitor is counted in one experience’s reach; in some personalization flows, a visitor may legitimately appear in multiple experiences over time.

### Experiments vs. Personalization views

- Experiment (A/B) report
  - One row for Original (control) and one for each Variation
  - Columns include Views, Reach, Goals hit, Revenue (and Profit if enabled), and comparison metrics (CVR, Revenue/visitor, or Profit/visitor)
- Personalization report
  - Splits results into “Personalized” vs. “Control” segments, with totals for each

### Statistical methodology (how significance and intervals are calculated)

The default and only methodology currently used in the UI is frequentist. Bayesian analysis is not available yet; it will be configurable in the future.

- **What we compare**
  - Conversion-level comparisons (CVR): compares proportions between control and a variation
  - Revenue/visitor and Profit/visitor comparisons: compares average monetary outcome per visitor between control and a variation
- **Frequentist methods (used today)**
  - CVR significance: two-sample proportion test (z-approximation). Inputs are visitors reached and goals hit for control vs. variation.
  - Revenue/visitor and Profit/visitor significance: two-sample test on means (normal/Welch’s approximation). Inputs are visitors reached and total revenue or total profit for control vs. variation.
  - Confidence intervals (CIs): per experience, a 95% CI is shown for the chosen metric (CVR, Revenue/visitor, or Profit/visitor). The box-plot visual shows the point estimate and CI band; overlap with control’s CI is shaded.
- Significance threshold
  - By default, a 95% threshold is used to indicate strong evidence of a difference.

### Improvement and “winner” logic

- Improvement (%): relative lift vs. control for the selected comparison metric.
  - Improvement = (Variation − Control) / Control × 100
- Winner suggestion: when viewing “All time” for the primary goal, the UI may suggest deploying a variation if it shows positive improvement and meets or exceeds the 95% significance threshold.

### Reading the experiment table

- **Views & Reach:** exposure volume (raw and unique)
- **Goals hit:** total number of attributed goal events
- **Revenue and Profit (if enabled):** totals attributed to the experience
- **Compare metric (selectable):**
  - **Goal CVR **— emphasizes goal frequency per visitor
  - **Revenue per visitor** — emphasizes average revenue per visitor
  - **Profit per visitor** — emphasizes average profit per visitor
- **Confidence: **the statistical significance (percentage) that the variation differs from the control
- **Confidence interval:** 95% range where the true metric is likely to lie for the chosen metric
- **Improvement:** relative lift vs. control for the selected compare metric

Tip: Switch the compare metric to match your primary objective (e.g., CVR for sign-ups; Revenue/visitor for revenue optimization).

### Historical trends

When you select a date range, charts group data into consistent time buckets (hour/day/month depending on range) in your chosen timezone and keep periods with no activity visible for continuity. This helps you see when performance changed and whether effects are stable over time.

### Profit analytics (Shopify only)

Profit metrics are available for Shopify stores when Profit Analytics is enabled for your account. Here is how profit is sourced and calculated:

- Data source: Shopify orders imported through the Shopify integration.
- Calculation: For each order, Profit = Subtotal (pre-shipping) − Cost of Goods Sold (COGS).
  - Subtotal is taken from the Shopify order subtotal (before shipping), in your store currency.
  - COGS is derived from Shopify Inventory unit costs for each purchased variant; if any line item lacks a unit cost or isn’t managed by Shopify inventory, profit for that order is not computed.
- Shipping: Shipping value is captured separately. In profit-based reports, you can optionally include shipping in “Profit per visitor” via a toggle in the UI.
- Availability: Profit appears only for Shopify stores and only when the Profit Analytics feature is enabled. If disabled or not applicable, profit columns and profit-based comparisons will be hidden.

### Practical examples

- CVR example: If a variation has Reach = 780 and Goals hit = 62, its CVR is 62 / 780 × 100 ≈ 7.95%.
- Revenue per visitor example: If the same variation has attributed Revenue = 14,500, then Revenue/visitor = 14,500 / 780 ≈ 18.59.
- Improvement vs. control: If control CVR is 6.0%, then improvement = (7.95 − 6.0) / 6.0 × 100 ≈ +32.5%.

### Best practices

- Define a primary goal and stick to it when judging experiments.
- Decide in advance on a minimum sample size or runtime before acting on results.
- Use Revenue/visitor when revenue is the primary objective; use CVR when the goal count is the focus.

### Glossary

- **Original (Control):** the baseline experience.
- **Variation:** an alternative experience used in personalization or A/B testing.
- **Reach:** unique visitors who saw an experience.
- **Goal:** an action you track (purchase, sign-up, custom event).
- **CVR (Goal Conversion Rate):** goals hit divided by reach.
- **Revenue per visitor:** revenue attributed divided by reach.
- **Profit per visitor:** profit attributed divided by reach.
- **Significance (Confidence):** how likely it is that a difference vs. control is not due to random chance.
- **Confidence Interval:** a range likely to contain the true metric value (95% by default).

### FAQ

- Why can Views be much higher than Reach? Are **Views** or **Reach** more important?
  - A single visitor can view the same campaign multiple times across pages and sessions (e.g., page reloads, returning visits), which increases Views. Reach counts each visitor only once. Views are helpful for understanding total exposure, but Reach is the primary metric used for comparisons (CVR, Revenue/visitor, Profit/visitor).
- Why does a variation with fewer views sometimes win?
  - Because comparisons are normalized per visitor (CVR, Revenue/visitor, or Profit/visitor), not by raw view counts.
- Can I include shipping in profit per visitor?
  - Yes. If profit analytics are enabled, you can toggle shipping to be included in profit-based comparisons in the UI.