The Incrementality Question Nobody in Ecommerce Is Asking (And How to Answer It Without a Data Science Team)
Here is the question that most DTC brands have never asked about their highest-spend marketing channel:
If we paused this channel completely for four weeks, would total business revenue decline by the amount this channel claims to generate?
In most cases, the honest answer is: we do not know.
And in a significant minority of cases, the answer — when brands finally test it — is: no. Not by anything close.
This is incrementality. It is the most important and most commonly ignored measurement concept in ecommerce marketing. And understanding it would change how most DTC brands allocate their budget.
WHAT INCREMENTALITY MEANS IN PLAIN LANGUAGE
Every marketing channel you run produces a reported revenue figure. Meta says it generated £40,000 this month. Google says it generated £25,000. Klaviyo says email generated £18,000. Add those up and you get £83,000 — significantly more than your actual Shopify revenue of £52,000.
The gap is attribution overlap. Multiple channels are claiming credit for the same customers. A customer who saw a Meta ad, later clicked a Google Shopping result, and then received a Klaviyo email before purchasing is counted as a conversion by all three.
Attribution models try to assign credit to one touchpoint or split it across several. But none of them answers the question that actually matters for budget decisions: which of these touchpoints, if removed, would cause that customer not to have purchased at all?
That question — which channel drives genuinely incremental revenue — is the incrementality question.
WHY IT MATTERS FOR BUDGET DECISIONS
If Meta is generating £40,000 of reported revenue and you pause it, two things can happen:
Scenario A: Total business revenue drops by £30,000–£40,000. Meta was driving genuinely incremental customers who would not have bought without the ad. The channel is worth what it claims to be worth (approximately).
Scenario B: Total business revenue drops by £8,000–£12,000. Most of Meta’s claimed conversions were customers who were going to buy anyway — driven by organic search, email, or word of mouth — and Meta was simply present in their journey and claiming credit via its attribution window.
In Scenario A, Meta deserves its budget allocation. Scale it confidently.
In Scenario B, Meta is consuming budget at roughly 3–4× its actual contribution. The optimal allocation looks very different.
THE HOLDOUT TEST — THE ONLY RELIABLE WAY TO MEASURE INCREMENTALITY
You do not need a data science team. You do not need sophisticated modelling. You need willingness to run a simple experiment.
Pause one channel for four weeks. Keep everything else constant. Measure total business revenue and MER before, during, and after the pause.
Practical considerations:
Choose a period with no major trading peaks. Do not run a holdout test over Black Friday or any significant promotional period — the external factors will confound the results.
Maintain other channels at constant spend during the pause. If you reduce Google spend at the same time as pausing Meta, you cannot isolate Meta’s contribution.
Measure at the business level. Use Shopify total net revenue, not platform-reported data. The incrementality question is about whether the business generates less money — not whether a platform reports fewer conversions.
Run the test for minimum four weeks. Short tests are unreliable — weekly variation in conversion behaviour can mask the true channel effect.
WHAT TO DO WITH THE RESULTS
If revenue declines significantly (within 20% of the channel’s claimed contribution):
The channel is genuinely incremental. Resume spending. The holdout test has validated your investment.
If revenue declines modestly (30–60% below the channel’s claimed contribution):
The channel is partially incremental. It does contribute something real, but it is significantly over-credited. Reduce its budget allocation to reflect actual contribution rather than claimed contribution.
If revenue barely declines (less than 30% of the channel’s claimed revenue):
The channel is primarily claiming attribution for conversions that would have happened anyway. This is the most important and most uncomfortable finding. It means a significant proportion of budget is being spent to show ads to people who were going to buy anyway — generating zero incremental value.
THE MER PROXY FOR BRANDS NOT READY TO RUN A HOLDOUT TEST
If pausing a primary channel feels too risky, there is a proxy method.
Scale a channel up by 20% for four weeks. Track whether total business MER improves proportionally.
If you spend 20% more and MER improves 15–20%: the channel is genuinely incremental at the margin. The additional spend is driving additional revenue.
If you spend 20% more and MER stays flat or declines: the marginal spend in this channel is not producing incremental revenue. You are spending more to claim the same conversions.
This test is less definitive than a holdout but provides directional evidence without the revenue risk of a full pause.
Book a free Growth Audit at exposegrowth.com/contact — we will assess which of your channels is likely to be genuinely incremental and which is primarily an attribution capture machine.