Through the perspectives of an experienced commissioner, and evaluator, this series explores the realities of evaluation Terms of Reference. Two voices – Jonathan Kuhn-Patrick (commissioner) and Mat Pritchard (evaluator) share their views in an open conversation.
Each article in the series examines the assumptions, tensions and trade-offs that shape evaluation in practice, drawing on decades of experience from both sides of the table.
On the polite fictions of the evaluation Terms of Reference
Every evaluation starts with a document
Mat and Jonathan: Every evaluation begins with a marriage of convenience. Two parties who do not entirely trust each other, and often do not even know one another, decide, for their own reasons, to join up.
And it begins, as these things do, with a document.
The Terms of Reference sets out what the commissioner wants from a partner, the kind of partner they want, what they will pay for it, and how long the whole thing should last. That is a perfectly reasonable document to need. It is also, more often than not, an unrealistic wish dressed up as a specification: everything the commissioner would like, costed as though they could actually have it. The bid that answers the advert makes its own hopeful case. Both sides can see the chasm between the two. They sign anyway.
Jonathan spent more than a decade commissioning evaluations from inside government and is now an independent contractor. Mat has spent his career on the other side, turning these documents into proposals that had to win and delivering evaluations when he did. Between us, we have written and read a great many, won some, lost others, and winced at more than we probably care to admit.
What follows is the part before the vows: how two parties who do not quite trust each other talk one another into it, and what each leaves unsaid. Later posts try to understand how we got into this in the first place, and then follow the marriage into less comfortable areas.
Our experience is in international development, because that is where we have spent our lives. But the pattern is not ours alone. Anyone who has commissioned or bid for an evaluation – in the health service, say, or the world of education or prisons – will recognise it. The setting changes; the moves do not.
The ToR that wants everything...
Jonathan: Let me start with the failing we commissioners least like to own.
We want everything.
We want rigour and depth. We want clear attribution of significant impact to our very modest intervention. We want every question answered and every stone turned, by the autumn, on a budget that would barely cover a mid-term review.
Deep down, we know the sums don’t work. But we tell ourselves the market will sort it out, that some clever soul will find a way and squeeze a small profit from the meagre budget on offer while delivering everything we ask for.
And we rarely stop to ask whether the thing we mean to evaluate can be evaluated at all: whether there is a baseline, a theory of change worth the name, or any usable data behind it. We are told there is, but we rarely take the time to really check.
...is actually the ToR that does not know what it wants
Mat: But I think “the Commissioner wants everything” is only half the story.
The deeper problem is that many Terms of Reference do not seem to know what they are for.
Yes, they ask for ‘everything’: every DAC criterion, every theme, every question anyone could think of. But that is often the surest sign of a commissioner who has not decided what they actually want.
That indecision is not free. The cost lands somewhere.
From the bidder’s side, it lands with us. We have to guess which two or three things you really care about (or should care about) and build the whole bid around that guess. Then we watch you mark the bid down for the things we did not lead with and which, quite possibly, we thought shouldn’t be important to you anyway.
A brief that will not commit to anything is not “leaving room for our judgement”. It is leaving us to carry the risk, and cost, of your uncertainty.
Jonathan: Mat is right, and I have written those briefs.
When you do not quite know what you want to learn, or cannot collectively decide, the safe-looking move is to ask for everything. So you reach for the full set of evaluation criteria, all of them, and wrap them in twenty or thirty questions, because a long and thorough-looking list feels responsible – and safe.
Naming the four things you genuinely need to know feels exposed. It invites challenge. It means telling colleagues that their particular issue, however important, may not be central to this evaluation.
And that is difficult. It is so much easier to accept requests for additions than to push back against technical advisers whose job is to ensure their particular issue is mainstreamed throughout the portfolio and who need evidence of that. The advisers are all incentivised to offer sage advice on what needs to go into an evaluation. It is their job. Rare is the adviser who reads a perfectly good ToR and says, “that is good enough, nothing from me.”
They always want their tuppence worth in.
The result proves the old line about a camel being a horse designed by committee. It wants everything, suits nobody, and hands the evaluator the job of deciding what the evaluation should really be for.
An evaluation built on sand
Jonathan: There is a third thing the ToR gets wrong, and it may be the most fundamental of the lot.
Evaluation is so often an add-on.
And the problem is not only timing — not only that a theory of change gets retrofitted onto a programme already two years into its life. It is that the foundations were never laid at all. No baseline worth the name. No monitoring data in a usable form. No logic that holds its weight when you lean on it. This is not a failing that a bigger budget or a longer timeline can put right. You cannot measure a change from a starting point nobody troubled to record.
And it falls squarely on us. The uncomfortable truth is that we rarely test whether a thing can be evaluated before we ask someone to evaluate it. We could commission an evaluability assessment — I have run one or two myself, from the consultant’s side — but time and money press against it, and so the ToR goes out taking evaluability on trust. We assume the data is there and usable, assume the baseline holds, and ask the evaluator to prove impact on that footing. Even the advisers whose job it is to ask the awkward question often don’t
Mat: And then we arrive, and sometimes find the foundations are sand.
We are asked to demonstrate impact against a baseline that was never taken, to judge effectiveness from monitoring data that was never collected, or collected and never usable. So we do what the evidence allows, which is usually less than anyone hoped for. And then – this is the part that stings — we are the ones who look thin. The commissioner is frustrated that the evaluator “can’t say anything meaningful”, when the reason is that there was never very much to say. The house was built on sand long before we were handed the keys.
Jonathan: The honest version would turn the order around. Work out whether the thing can be evaluated at all, and put right what is missing, before deciding what to ask of it. Foundations first, questions second.
A ToR built on rock can ask a great deal. A ToR built on sand cannot safely ask anything. We keep writing the second and hoping it behaves like the first.
The bidder’s gamble
Mat: This is where the silence starts.
The evaluator is often the person best placed to say the difficult thing: the budget will not stretch; some of these questions can only be partially answered, if at all; there are too many questions; the baseline is not there; the timetable is wildly optimistic.
But the evaluator is also the person with the strongest incentive not to say it too clearly.
Say it in your bid and you look difficult. Or negative. Or not up to the job. Or insufficiently “solutions-focused”.
And the contract goes to whoever said it all sounded fine.
So nobody quite says it. Not directly. Not in the way they might say it over coffee, or to a trusted colleague, and certainly not in the bid, which is the one place it would count.
Some serious players may simply not bid. But for those who do, they’re gambling that the unspoken doubts can be surfaced later, at a point we will come to in a later post, once the balance of power has shifted.
Jonathan: From the commissioner side, this is uncomfortable, because Mat is describing something many of us know perfectly well.
A bid full of caveats may be the honest bid. But it may not look like the strongest bid. It may look cautious, awkward, underpowered, or lacking confidence. The bidder who says “we can do that” is easier to appoint than the bidder who says “you need to rethink this.”
That does not mean commissioners are fools. We know there are trade-offs. We know the budget has limits. We know that some questions are harder than others. But procurement is not always designed to reward the most honest diagnosis of the problem. It is often designed to reward confidence, compliance, and the appearance of control.
Mat and Jonathan: The uncomfortable truth is that both sides are reading the fiction, and both sides know it is fiction, but the incentives are not always aligned around saying so. So the doubts get carried forward. Into the bid. Into the contract. Into the inception phase. Into delivery. And by then, of course, they are much harder to deal with.
Risks noted, not mitigated
Jonathan: Even the people we ask to check cannot always save us.
Quality assurance of a terms of reference, where it happens at all, can advise. The reviewer can see that a brief looks overloaded, a budget looks thin, a timetable looks optimistic, or an evaluation question is too vague to be answered. They can say the risks are high. They can recommend changes.
But they cannot always force those changes.
Part of the problem is distance. The reviewer is independent precisely because they stand far enough back not to be caught up in the programme. That helps them see some things clearly. It also means they may not know the politics, history, constraints, personalities, or organisational compromises that produced the ToR in the first place.
So they note the risks. And the ToR goes ahead regardless.
Mat: This, to me, is one of the subtlest fictions of all: the idea that noting a risk is the same as managing it.
It is not.
If the overall system simply absorbs that risk warning and carries on, then QA has become a kind of ritual. Better than nothing, possibly. But not the same as mitigation.
The problems emerge later, as expectations harden. Bidders make decisions, spend money, envelop the fiction in a web of assumptions that is often quite hard to unpick later on.
The warning signs are there. But the machine keeps moving.
So why do we keep signing?
Mat and Jonathan: None of this is an argument against going ahead.
We do not live in a world of perfect briefs or honest bids, and a marriage of convenience entered with open eyes can be a good one. Plenty of the working relationships either of us has most valued began exactly that way.
ToRs are not bad because commissioners are foolish. Bids are not flawed because bidders are uniquely dishonest. The problem is more uncomfortable than that.
The system incentivises wishful documents, strategic bids, deferred honesty and risk-shifting. It rewards the appearance of certainty before certainty has been earned. It asks commissioners to specify more than they can know, and bidders to promise more than they can safely deliver.
The grown-up move is not to hold out for perfection, because perfection really is the enemy of the good. It is to stop pretending the wish is the plan. It is to name the shortcomings, on both sides, before signing rather than after.
Only when you recognise failings can you address them. A system blind to, and incurious about, its own failings has no hope of addressing them.
Which leaves the question the whole thing raises.
If everyone can see the wish is half-fiction, and everyone signs anyway, why do we keep doing it? What is in our collective DNA that leads us repeatedly down this path, towards a dishonest marriage of convenience?
That is the subject of our next article in this series.
A note on AI. Both authors used AI tools to help draft and edit this post. We did so in line with the UK Evaluation Society’s good practice guidance on the use of AI in evaluation: with human direction throughout, and with the two of us taking full responsibility for the final text and the views in it.
Our thanks to Jonathan Kuhn-Patrick and Mat Pritchard for this guest blog and for contributing their perspective to the UK Evaluation Society’s work.
How to cite this article
Kuhn-Patrick, J. and Pritchard, M. (2026) Terms of Endearment: On the polite fictions of the evaluation Terms of Reference. UK Evaluation Society article. Available at: https://evaluation.org.uk/article-terms-of-endearment/ (Accessed: [insert date]).
Terms of Engagement. Jonathan Kuhn-Patrick and Mat Pritchard examine how procurement shapes evaluation, transferring risk, influencing behaviour and creating hidden costs.
About the authors
Jonathan Kuhn-Patrick
Jonathan Kuhn-Patrick is an Independent Evaluation Consultant and international development and evaluation specialist with more than 30 years’ experience spanning humanitarian response, fragile states and complex programme evaluation. His career has included frontline NGO leadership, senior evaluation, policy and operational roles within DFID and FCDO, and extensive work across Africa, the Middle East and South Asia.
Now an independent consultant, Jonathan specialises in evaluability assessments, evaluation quality assurance and monitoring, evaluation and learning (MEL) framework design. His recent work has focused on fragile states, humanitarian response, climate resilience and infrastructure transparency across Asia, Africa, the Middle East and Latin America.
Jonathan led the development of the UK Evaluation Society’s good practice guidance on the use of AI in evaluation and continues to support professional learning in this area. He is also a former member of the UK Evaluation Society’s Board of Trustees.
Mat Pritchard
