How Incrementality Testing Can Save Your Media Budget

Marketing teams have been applying scientific principles to solve business problems for more than 100 years. They have continuously refined the processes of data collection, analysis, and experimentation to better understand and predict customer response. As marketing and commerce moved from brick-and-mortar to digital channels, more tools and use cases for data-driven marketing emerged. Still, the core focus remained the same: read and respond to market conditions, and continually strive to understand and influence customer behavior.

Spend is a powerful driver of customer behavior and is fully within the marketing team’s control. As marketers scale spend, they expect to see tangible changes in KPIs such as brand awareness, traffic, customer acquisition, and sales. While business intelligence tools track changes in these KPIs, isolating the additive impact of spend is more difficult. This impact has been labeled incrementality, and it matters because it signals true growth.

Incrementality is a uniquely powerful input for budgeting and building a business case for investment. Insights into the incrementality of ad spend by channel or campaign give marketers greater control over outcomes and are directly linked to a brand’s growth. For example, a channel that delivers $30 ROAS might deliver $0 iROAS if it heavily overlaps with other channels and over-indexes on existing customers. Conversely, a channel that delivers $5 ROAS may also deliver $5 iROAS if it is reaching a new audience and driving net-new customers. In other cases, companies may observe low iROAS but highly incremental traffic, a positive signal when that traffic is “filling the funnel” with new leads expected to convert in the future.

The most effective way to quantify incrementality is through an in-market test, also called a controlled experiment. While experimental designs vary, all incrementality tests share a core feature: they control for spend to compare actual results against a baseline representing what would have happened without the ad spend. Any difference in outcome between the experimental group and the control group is attributed to that spend and defined as incremental.

Marketers are already skilled at reading results and making decisions. While incrementality tests are interpreted in absolute terms, their application must be evaluated relative to other channels and tactics across the media mix. Incrementality test results typically inform:

1). Media Mix Models (MMM)

Incrementality tests act as a “Bayesian prior” for MMMs. Because MMMs are models, they are easier to implement than in-market tests but are also more prone to error. MMMs must be validated by in-market tests to be considered a reliable source of truth. Given the widespread use of MMM solutions across retail companies, every launched incrementality test should be expected to inform an MMM.

2). Attribution Hygiene

Companies relying on last-touch or multi-touch attribution sometimes apply an “incrementality factor” to adjust reported revenue and orders, and better reflect a channel’s perceived contribution. This approach is highly subjective and often creates more chaos than consistency in reporting.

3). Performance Management

Tests are typically run 2 or 3 times per year as brands seek to both pressure-test market response and evaluate how operational improvements in channel marketing have affected ROI. It’s highly unlikely that a channel would be affected after a single incrementality test. Rather, it’s more likely that leadership will evaluate the channel  for growth potential and re-test at a future date. When underperformance is attributed to execution rather than the channel itself, leadership may reassign a channel to a different team in an effort to improve results.

4). Media Mix Changes

Marketing teams are generally risk-averse and unlikely to make significant changes to the media mix without rigorous validation. Incrementality tests clarify the impact of shifting funds across the funnel (e.g., from upper- to lower-funnel) or within a funnel stage (e.g., reallocating among upper-funnel tactics).

5). Budget Allotments

Marketing budgets are a percentage of sales and thus are tightly linked to the company’s P&L. During re-budgeting cycles, teams may be required to reduce spend. Recent incrementality tests provide a critical data point for “right-sizing” channel investment. Cutting spend in a channel or campaign with low incrementality is less risky than cutting one with high incrementality. It is also a reminder that decisions to increase or decrease spend may be driven less by channel performance and more by broader macroeconomic or company-level factors.

‍

In conclusion, the incrementality of a channel is a key metric for marketing teams to understand both past performance and set proper expectations for the future. The metric itself can only be fully evaluated as good, bad, or neutral relative to the performance of other channels, the broader business context, and the specific goals set for the team.