Author: Roy Aulie Jacobsen, PhD research fellow at Kristiania University of Applied Sciences.
In this guest blog, Roy Aulie Jacobsen, offers two simple, usable safeguards for commissioners and evaluators: a three-sentence “stress test” (Observed / Interpretation / Boundary) to keep claims proportionate, and a “claim lock” sentence agreed at sign-off to prevent accidental overclaiming in downstream products.
Keeping evaluation claims honest when reports become slides
Evaluation is often judged on design: was the sample adequate, was the comparison credible, was the analysis sound? Those questions matter. Yet a quieter test of quality arrives later, when evidence is turned into a claim that has to survive real decisions.
A finding rarely stays where it was produced. It moves from report to summary to slides to public messaging. Each step rewards compression, and compression can strengthen the claim. The risk is not usually deliberate spin. It is claim drift: language becomes more confident as it becomes shorter.
Consider the difference between “participants reported higher confidence over the delivery period” and “the programme improved confidence”. The first stays close to an observation. The second implies a counterfactual. In a busy meeting, those sentences can sound interchangeable. They are not. It is often at this point that the room goes quiet, because everyone hears the difference, but the timetable still demands a decision.
Why drift happens
Claim drift is a predictable organisational dynamic. Decision urgency rewards decisive wording. Accountability settings reward statements that look settled. Reputational risk makes ambiguity feel unsafe, because uncertainty can be misread as weakness. Under those pressures, qualifiers fall away, and the strongest-sounding version is the one most likely to be repeated.
This is why overclaiming is not only a writing issue. It is professional judgement with ethical consequences. Funding choices, service changes, and public trust can be shaped by rhetorical certainty rather than defensible inference.
A tool that works: the Three-Sentence Stress Test
One practical way to reduce drift is to create wording that stays true when shortened. Write three sentences that you would be comfortable seeing in a short brief read by someone outside the project. The first states what was observed, staying close to the evidence and the evaluation’s scope, without quietly importing causality. The second offers the most defensible interpretation for decision-making, naming a plausible mechanism only where the evidence supports it. The third states what cannot be concluded from the current evidence, and what would be needed to claim more.
- Observed: “Attendance increased during the delivery period, and staff accounts suggest fewer missed appointments.”
- Interpretation: “The pattern is consistent with improved accessibility, but other changes (such as referral practices) may also have contributed.”
- Boundary: “Without a credible counterfactual, or stronger process evidence on alternative explanations, it is not possible to conclude that the programme caused the change.”
The point is not to make conclusions timid. It is to make them stable, so they can travel across formats without quietly becoming stronger. This is especially useful for snapshot evaluations, pilots, and rapid reviews.
A second safeguard: the claim lock
The stress test works best if it becomes the anchor for everything that follows. A simple safeguard is to agree a “claim lock” at sign-off: one sentence that must appear verbatim in every downstream product, from summaries and slides to external communications. In practice, this is often the boundary sentence, because it is the first thing to be edited out when space is tight and the most important thing to preserve when decisions carry risk.
Commissioning well: make the claim contract explicit
This discipline is easier when commissioning makes the “claim contract” discussable early. Many evaluations implicitly ask for attribution, because attribution settles arguments and justifies budgets. The contract, however, often only buys evidence that can support description or a careful contribution story. If that mismatch stays unspoken, it will usually reappear late, as pressure for a headline that the evidence cannot carry.
A simple commissioning question helps: what kind of claim is needed for the next decision, and what kind of evidence would be required to support it responsibly? UK guidance, including HM Treasury’s Magenta Book, emphasises proportionate, fit-for-purpose choices (HM Treasury, 2020). The practical implication is that the claim should be proportionate too.
Quality in the age of circulation
Evaluation is a social practice, not only a technical one. If findings move through interconnected audiences and accountability settings, then quality cannot live only in method. It also lives in the disciplined link between evidence and claim. Classic work on evaluation use has long argued that findings are interpreted and mobilised in organisational contexts, rather than simply “applied” as neutral facts (Weiss, 1979).
The simplest marker of discipline is whether the conclusion remains honest when it becomes short. If the three-sentence translation cannot be written without either overclaiming or hiding uncertainty, the problem is not the audience. It is the claim. The claim is the intervention’s shadow. If it stretches further than the evidence, evaluation stops being a public good.
Our thanks to Roy Aulie Jacobsen for this guest blog and for contributing their perspective to the UK Evaluation Society’s work.
Roy Aulie Jacobsen
PhD Fellow in Political and Crisis Communication | YECREA Representative – ECREA Mediatization & Society Section | Data-Driven Insights | Strategic Messaging & Framing
Roy Aulie Jacobsen is a PhD research fellow working at the intersection of political communication, political science, and sociology, with a primary focus on strategic messaging from political leaders and their crisis response. His work centres on understanding how political actors communicate under pressure—whether during elections, public health emergencies, or high-stakes policy reforms.
