What we track
A Shopify Web Pixel and the blocks themselves record what happens on your store: an impression for each product page view where Brummell shows, a view for each block that comes on screen, add to cart, checkout started, purchase and refund, plus try-on and outfit editor events where you have switched those on. Each event rolls up per funnel stage, so a result can be traced back to where in the funnel it happened, not just an end-of-month total.
The metric
The primary metric is add-to-cart rate on product pages that can show an outfit. Purchase rate is reported beside it as the secondary metric. The reason is arithmetic: at a small store, purchases are too rare to separate a real change from noise within 90 days, while add-to-carts happen often enough to answer the question.
The holdout group
Every visitor is assigned to an arm, exposed or held out, deterministically and once. The assignment is stable across sessions: the same visitor lands in the same arm every time they return, so results are not diluted by people flipping between groups.
Holdout is enforced, not just recorded. For a holdout visitor, the underlying placement code skips rendering entirely, it is not merely logged as a control-group impression after the fact. That distinction matters: a holdout number is only real if the widget genuinely never rendered for that shopper.
How the holdout is sized
The holdout share is not a fixed percentage. It is calculated per store from that store's own baseline conversion rate, a target minimum detectable effect, and a 95% confidence level, using a standard two-proportion sample-size formula. In practice this keeps the holdout dynamically bounded between 5% and 20% of traffic: wide enough on a low-traffic store to reach a real answer, narrow enough on a high-traffic store that you are not leaving revenue on the table longer than necessary.
When a result counts as significant
A lift number is not surfaced until the calculated sample size for that store has actually been reached. There is no fixed calendar window behind that gate, a low-traffic store and a high-traffic store each need a different amount of time, and the dashboard reflects real progress toward that number rather than a countdown that is the same for everyone.
While it collects: the Overview, day 2
Once the sample is big enough: example data, not a store's result
Measured counts, beside the comparison
Under the comparison result, Impact also shows what Brummell can count directly. Revenue from Brummell adds this month is the value of the cart lines a Brummell block added, after discounts, in orders placed this calendar month. Each of those lines carries a hidden cart property, so the count is exact, and it is labelled for what it is: measured, not a lift estimate. Where try-on is on, a Virtual try-on card covers the last 30 days: shoppers who opened it, try-ons made, add-to-carts after a try-on, and the orders that followed.
Brummell asks for a review only at a measured milestone: when the comparison has confirmed a positive add-to-cart result, or when 10 orders in a month included a piece added from a Brummell block.
Returns count too
A refund webhook tracks the 30-day return rate separately for the exposed and holdout arms. A placement that drives orders which are returned at a higher rate is not actually driving the lift it appears to on a gross-revenue view, so return rate is part of the same measurement, not an afterthought.
Returns are a measured arm, not a promise. The idea that a piece a shopper can see in several outfits is a piece they keep is a hypothesis this measurement tests on your store; we do not claim fewer returns before your own data shows it.
Why we built it this way
Most merchandising widgets report a lift number with no control group behind it, which makes the number unfalsifiable. Ours is built so the holdout group is real, the significance check is tied to your store's own traffic instead of a one-size-fits-all timer, and returns are weighed against the same lift they helped produce. That is the whole methodology. Nothing here depends on a claim you cannot check against your own dashboard.