What we track
A Shopify Web Pixel captures five events as shoppers move through your store: product view, Brummell widget view, add to cart, checkout start, and purchase. Each event rolls up per funnel stage, so a lift claim can be traced back to where in the funnel it happened, not just an end-of-month total.
The holdout group
Every visitor is assigned to an arm, exposed or held out, deterministically and once. The assignment is stable across sessions: the same visitor lands in the same arm every time they return, so results are not diluted by people flipping between groups.
Holdout is enforced, not just recorded. For a holdout visitor, the underlying placement code skips rendering entirely, it is not merely logged as a control-group impression after the fact. That distinction matters: a holdout number is only real if the widget genuinely never rendered for that shopper.
How the holdout is sized
The holdout share is not a fixed percentage. It is calculated per store from that store's own baseline conversion rate, a target minimum detectable effect, and a 95% confidence level, using a standard two-proportion sample-size formula. In practice this keeps the holdout dynamically bounded between 5% and 20% of traffic: wide enough on a low-traffic store to reach a real answer, narrow enough on a high-traffic store that you are not leaving revenue on the table longer than necessary.
When a result counts as significant
A lift number is not surfaced until the calculated sample size for that store has actually been reached. There is no fixed calendar window behind that gate, a low-traffic store and a high-traffic store each need a different amount of time, and the dashboard reflects real progress toward that number rather than a countdown that is the same for everyone.
Returns count too
A refund webhook tracks the 30-day return rate separately for the exposed and holdout arms. A placement that drives orders which are returned at a higher rate is not actually driving the lift it appears to on a gross-revenue view, so return rate is part of the same measurement, not an afterthought.
Why we built it this way
Most merchandising widgets report a lift number with no control group behind it, which makes the number unfalsifiable. Ours is built so the holdout group is real, the significance check is tied to your store's own traffic instead of a one-size-fits-all timer, and returns are weighed against the same lift they helped produce. That is the whole methodology. Nothing here depends on a claim you cannot check against your own dashboard.