What Research Says About Benchmarking KPIs Against Competitors

Almost every digital marketing planning guide gives the same instruction: before you set objectives, review your own performance benchmarks and your competitors'. It's standard advice, repeated widely enough that it rarely gets examined.

It's worth examining, because the empirical research on benchmarking contains at least one finding that contradicts how the advice is usually given — and a distinction that most practitioner guidance skips entirely.

The instruction combines two management ideas developed separately, in different fields, about a decade apart.

SMART objectives come from George T. Doran's article in the November 1981 issue of Management Review, "There's a S.M.A.R.T. way to write management's goals and objectives" (vol. 70, no. 11, pp. 35–36). Two details are routinely misremembered. Doran's criteria were Specific, Measurable, Assignable, Realistic, Time-related — the "A" meant assignable, naming who owns the objective, not "achievable." And Doran was deliberately undogmatic: he noted that quantification isn't feasible at every level of management and that not every objective needs to satisfy all five criteria. He presented SMART as a thinking checklist, not a compliance standard. He presented no empirical study; the article drew on his experience in corporate planning. [Doran 1981]

SMART can tell you whether an objective is well-formed. It cannot tell you whether the number is ambitious, complacent, or fantasy.

The framework tells you whether an objective is well-formed. It cannot tell you whether the number you picked is ambitious, complacent, or fantasy. Nothing internal to SMART supplies that.

Benchmarking supplies it, and comes from elsewhere. Robert Camp introduced the practice at Xerox and codified it in Benchmarking: The Search for Best Practices That Lead to Superior Performance (ASQC Quality Press, 1989), later extended in Business Process Benchmarking (1995). Camp defined it as a structured process for identifying and replicating best practices to improve business performance. [Camp 1989; Camp 1995] By 2001 it was among the most widely used management tools in the world, central to total quality management, knowledge management, and process improvement work. [Rigby 2001]

Camp's process has three stages, as later summarised in the marketing literature: a search stage, in which managers look for firms showing superior performance and identify the capability drivers behind it; a gap-assessment stage, in which differences between the firm and the benchmark are measured; and a capability improvement stage, in which gap-closing changes are planned and executed. [Vorhies & Morgan 2005, p. 81, summarising Camp 1995 and Garvin 1993]

For a long time this was normative advice with little evidence behind it. Despite widespread encouragement to managers, the benchmarking of marketing capabilities as a route to competitive advantage had received almost no empirical attention. [Vorhies & Morgan 2005, p. 80]

Douglas Vorhies and Neil Morgan tested it in the Journal of Marketing (2005, 69(1), 80–94), surveying top marketing executives at 748 US firms across twelve industries and getting 230 usable responses, a 31% response rate. They validated the perceptual results against objective two-year-average return-on-assets data for a 109-firm subset. [Vorhies & Morgan 2005, pp. 82–83]

This isn't a fringe position. Camp had argued in 1989 that restricting benchmark searches to your own industry reduces effectiveness, partly because many industries contain few major firms and those firms tend toward mimetic isomorphism — they've converged on each other's practices, so there's little left to learn. [Camp 1989, as discussed in Vorhies & Morgan 2005, p. 83]

On how many benchmarks to use, more was not better. Single, top-five, and top-eight benchmark groups performed near-identically (R² = .21, .20, .19; β = −.44, −.43, −.42), but at sixteen firms both model fit and effect size declined (R² = .16, β = −.39). [ibid., Table 3] Widening the net dilutes the benchmark toward the mean.

On what to benchmark, the eight marketing capabilities they measured — product development, pricing, channel management, marketing communications, selling, market information management, marketing planning, marketing implementation — were interdependent, and the indirect paths running through that interdependence were stronger than the direct paths from each capability to performance. Their conclusion: these should be benchmarked as a set, not individually. [ibid., Figure 1, p. 85] Weighting individual capabilities by their performance contribution added nothing over the unweighted model. [ibid., Table 2]

Vorhies and Morgan observe that the focus of benchmarking shifted over time — away from the content of products and services, the strategies pursued, and the performance outcomes achieved by top performers, and toward the capabilities believed to have produced those outcomes. [Vorhies & Morgan 2005, p. 81]

The digital marketing practice of benchmarking KPIs is largely the older form. Click-through rate, conversion rate, cost per acquisition — these are outcome metrics. Knowing a competitor converts at 4% and you convert at 2% tells you a gap exists. It tells you nothing about which capability produced the difference, and therefore nothing about what to change.

Vorhies and Morgan's own recommendation runs the other way: managers should first identify the capability drivers of superior performance, then assess the gap on those, rather than allocating improvement resources to whatever capability feels internally weak. [ibid., p. 90] By focusing on capability sources rather than observed outcomes, and using competitors and peers as referents, benchmarking becomes part of a genuine marketing control system. [ibid., p. 89, drawing on Day & Wensley 1988 and Morgan, Clark & Gooner 2002]

KPI benchmarking is best understood as diagnostic rather than prescriptive. It locates a gap; finding what's behind the gap is separate work.

Which suggests KPI benchmarking is best understood as diagnostic rather than prescriptive. It locates a gap. Finding out what's behind the gap is separate work, and it's the work that actually sets an achievable target.

Benchmarking data isn't neutral, and the marketing-audit literature makes this point sharply: much of that literature assumes the data an auditor collects is objective, when in fact information about organisational activity is never neutral — particularly in the context of performance appraisal, where audits typically gather opinions and accounts from informants positioned in different parts of an organisation. [Brownlie 2000, Long Range Planning] For external competitor benchmarking the problem compounds, since competitors don't publish internal metrics and inferring them from public signals imports estimation error into your target.

The measurement literature also offers no settled framework. Reviews note that there's no universal tool for marketing performance measurement, in part because of the sheer abundance of available metrics, and that authors frequently don't state the criteria by which they selected the metrics they study. [Gao, "Measuring Marketing Performance: A Review and A Framework"]

Vorhies and Morgan were candid about their own constraints: they benchmarked eight mid-level capabilities, which precluded assessing higher-level integrative capabilities like brand management or CRM; they couldn't control for firms' non-marketing capabilities such as R&D; and their measures were broad enough to lack depth on any single capability. [Vorhies & Morgan 2005, pp. 91–92] Their cross-industry finding is specific to their sample — they explicitly say managers should assess for themselves whether it holds in their own industry and strategy context. [ibid., p. 90]

The instruction to benchmark before setting objectives is sound, and there's now empirical support for it. But three common refinements of it are worth revisiting: