Through the perspectives of an experienced commissioner, and evaluator, this series explores the realities of evaluation Terms of Reference. Two voices – Jonathan Kuhn-Patrick (commissioner) and Mat Pritchard (evaluator) share their views in an open conversation.
Each article in the series examines the assumptions, tensions and trade-offs that shape evaluation in practice, drawing on decades of experience from both sides of the table.
New to this series? We recommend starting with Terms of Endearment, where Jonathan and Mat explore why evaluation Terms of Reference often become a "polite fiction" and set the scene for the discussion that follows.
From polite fiction to risk transfer
Mat and Jonathan: In the first post, we argued that the evaluation Terms of Reference is often a polite fiction. Many ToRs ask for too much, decide too little, assume evaluability rather than test it, and invite bidders into a shared performance of confidence. The commissioner asks for the thing they would like. The bidder describes the thing they can just about make sound possible. Both sides can see the gap. Both sides proceed. Which left us with the question we ended on last time: if everyone can see this, why do we keep doing it? What is in our collective DNA that keeps taking us down the same path?
Jonathan: I’ll start with a defence from the commissioner’s side, because procurement systems exist for good reasons. They are not designed to make anyone’s life difficult. But public money cannot be handed out on the basis of who the commissioner happens to know – and commissioners know a lot of evaluators – or on instinct, or a gut feeling, or any other reason that won’t survive daylight. The process has to be fair to bidders, transparent to auditors, defensible to senior management, robust enough to survive challenge, and still deliver the right result: a contract that does what’s needed at good value for money. At its best, procurement protects both the commissioner and the market. It puts order around what would otherwise be a messy, subjective and highly contestable decision.
Mat: All true. But the procurement system does more than select a supplier. It moves risk around. The commissioner’s uncertainty, which we discussed in the first post, becomes a risk transferred into the bidder’s design. The fixed budget becomes the supplier’s commercial gamble. The cost of failed bids never appears on the commissioner’s spreadsheet, but it is invisibly costed into every bid anyone submits. And it isn’t only commissioners who transfer risk – bidders do it too. The parts of the ToR that were glossed over, or delicately fudged, don’t go away. They get carried into inception and delivery: the can kicked down the road for someone else to sort out later. That is the hidden part of procurement. It is not only about how risk is managed, but how it is moved, and onto whom.
How the budget becomes the anchor
Jonathan: It’s worth taking the budget first, because it is usually the first – and most important – hard number that all the polite fictions arrange themselves around. The budget for an evaluation is not plucked from thin air (well, not usually.) But it is often built from imperfect assumptions and data and settled through a contested process of trade-offs. It may be shaped by a rule of thumb – some percentage of the programme budget – by reference to previous contracts, or simply by what survived an internal competition for funds. The specific context of the evaluation often isn’t fully understood, or isn’t allowed to change the number. And there are hard limits: the money available, internal ceilings, shifting spending pressures. Commissioners also misread day rates. A rate looks big, so they assume there’s slack in it. There usually isn’t.
Mat: I’ve suspected all of this. And the budget is usually the least flexible part of the ToR, so it starts shaping the bid immediately. The method has to be designed to fit the budget, not the other way around. More than that: most firms know they need to come in under budget to look competitive, so they discount before they’ve started. We work hard to fit the approach to the money. That is one of the largest risk transfers in the whole process – the greater the ambition and the more modest the resources, the more risk moves across.
The costs commissioners don’t see
Mat: That leads to the next hidden cost: bidding itself. Bidding is real work, and it costs real money – senior technical design, methodology, pricing, partner management, chasing CVs, editing, quality assurance. A large bid can cost a firm tens of thousands of pounds. Even a small one is rarely cheap, because so many of the fixed elements are still required. Win rates matter here: winning somewhere between one in three and one in five is usually considered healthy, and much below that is hard to sustain, because every successful bid must pay for itself and for all the ones that failed.
Jonathan: Some regimes pay an honorarium for shortlisted design work, and you could argue evaluation should follow. I’d resist it, for most evaluation work. Pay people to bid and you invite bids from those with no real intention of delivering. The discipline of unpaid competition is doing something useful, it’s just doing it unevenly, with small firms and hopeful associates paying the price of working up a bid for nothing. None of which makes procurement cost-free. The win rate is simply another risk transfer, settled in unbilled hours.
The market we say we want
Jonathan: This is a market-related point that commissioners don’t always see, because from where they sit the machinery is working. A ToR goes out, the tender goes live, bids arrive, a winner emerges. Job done. But it deserves more thought. We say we want a competitive, efficient market – a wide field of large, medium and small organisations, with innovation and specialist skills, so we aren’t forever dependent on the same few big suppliers. That’s the argument for “widening the field”. Yet the more it costs to bid, the fewer bid failures a small organisation can absorb.
Mat: And “widening the field” needs unpacking, because it isn’t one aim, it’s several – and they pull against each other. More bidders? Better bidders, with more innovation and less market capture? Or simply lower prices? They are not the same thing. More bidders doesn’t mean better value; it means more failed bids, which raises the cost base of the whole market, because every winning contract has to absorb more unpaid work. Smaller and specialist firms can’t carry that bidding risk as easily as the big ones. So a drive to widen the market can actually narrow it, favouring exactly the large organisations that can afford to lose.
Jonathan: In fairness, the regulators have noticed. The Procurement Act now places a duty on authorities to reduce barriers for smaller suppliers, and mandates 30-day payment down the supply chain – both aimed squarely at the narrowing we’re describing. The aim is right, and I’d defend it. But I’ll be frank about its limits: a payment-terms rule doesn’t touch the economics of unpaid bidding. You can pay a small firm promptly for the work it wins and still price it out through the work it doesn’t.
And this isn’t only our impression. Molly Sundberg’s 2024 study of consultancy procurement at Sida, the Swedish aid agency, found the same clear pattern: framework agreements favoured large firms so consistently that, over fifteen years, small staff-owned firms went under, merged, or were bought up, leaving a market dominated by two foreign-owned companies. The machinery built to widen the field measurably narrowed it.
Frameworks: tidier, not necessarily cheaper
Jonathan: Commissioners have tried to ease some of these tensions. Frameworks, for example, earn their keep. They pre-vet suppliers, cut compliance time, and give a commissioner a defensible pool without running a full open competition every time. As a commissioner, I’d defend them.
Mat: They matter to suppliers too, especially when a real flow of work runs through them. But they don’t escape the law of unintended consequences, and their attraction is often the access they give rather than any saving on the cost of bidding. Start with the obvious: getting onto a framework is itself a major bid – full technical and commercial effort – and then you bid again for every call-off. That’s two competitions, not one. I’ve never much liked the term “mini-competition”, either. Almost every framework call-off I’ve worked on was as demanding as an ordinary open tender; the “mini” is in the paperwork, not the effort. Any supplier who treats one as light-touch loses to those who don’t, so everyone puts the effort in regardless. And the initial bid to get on a framework is a gamble on a pipeline: some produce a steady stream of work, others go quiet, and if little comes through, you never recover the cost of getting on. Meanwhile the deeper problems are untouched – frameworks don’t tidy away over-ambitious ToRs, unclear priorities, hidden bid costs, or risk pushed down the chain.
Jonathan: There’s a commissioner-side irony in that, too. Admit enough consortia to a framework, and a single call-off can still attract a dozen bids. At which point it’s worth asking how different it really is from the open competition it was meant to streamline.
The individual face of risk transfer
Mat: And at the bottom of all this sits the individual consultant, carrying risks they are least able to manage. Procurement demands named experts, and asks them to confirm availability at the bidding stage – then behaves as if that provisional yes were a firm commitment. But only one bid in a field can win, and consultants have bills to pay; they can’t hold large blocks of time open indefinitely. The system acts as though they can. The scale of it is easy to miss: if ten consortia each name ten experts for a role, a hundred people are pencilled in for a job one of them will do. Most were never going to get it, and they know it. And the timetable stretches the fiction further – a tender drops in June promising a late-August start, and the start rarely comes when promised.
Often the hold is exclusive, too. A firm will lock up a named Team Leader – to help write the bid, or at least to quality-assure it, and to stop them carrying the approach to a rival – so it isn’t just time held open, it’s time held for one bidder alone. And “legally binding availability” is largely a fiction: who is realistically going to sue an individual consultant, and how would you compel someone to be available and then expect good work from them? The real question is the ethics of loading that risk onto the people least able to absorb it. I’ve cleared a diary for a bid that slipped from summer to the following spring and then never landed at all. That gap appears on no one’s risk register but mine.
Jonathan: And the evidence bears this out. Sundberg found consultants earning more working directly for the agency than as sub-contractors on the same kind of work once frameworks took hold – the lead firm takes the margin – and concluded that framework agreements raised procurers’ costs while cutting individual consultants’ incomes. She also describes competition, under frameworks, not disappearing but descending: from a contest between buyer and vendor to one between vendors – consortium partners, sub-contractors and freelancers, all competing among themselves. That descent ends exactly where we’ve been pointing: on the individual.
What procurement rewards
Jonathan: All of which raises the larger question – what does procurement actually reward in practice? We know what we say we want: comparable bids, a quality team, value for money, compliance for the sake of transparency and audit, and credible delivery. But in practice the system can also reward confidence and the appearance of control where control isn’t really possible; light caveating, where a few honest caveats would be worth a great deal; aggressive pricing, where everyone suspects the resources fall short; and CVs that reassure – which often means the same faces we’ve seen before.
Mat: Exactly – and comparison and competition can punish the bidder who tells an uncomfortable truth or caveats honestly. Some of that is baked into the scoring: technical and commercial scores are weighted and combined, and on plenty of evaluations the commercial weighting is heavy enough that a confident underbid beats a properly costed one.
The Procurement Act’s headline change – from the Most Economically Advantageous Tender to the Most Advantageous Tender, MEAT to MAT – is meant to loosen price’s grip and let “value” mean more than cost. Whether that changes behaviour or merely relabels it is an open question. Either way, the difficult conversations get deferred to inception – which is precisely where the risk we’ve been tracking finally comes due. But that’s a conversation for the next part of this series.
So where does this leave us?
Jonathan and Mat: This isn’t just procurement-bashing. There are few if any perfect processes, and procurement is necessary; fairness, transparency, accountability, defensibility and value for money are not optional. But procurement is not a neutral gateway between the ToR and delivery. It shapes what suppliers say and what they gloss over, and it shapes who can afford to compete. Most of all, it governs risk – what is accepted, what is transferred and to whom, and what is quietly deferred:
- commissioner uncertainty becomes bidder cost;
- bid failure becomes supplier overhead;
- supplier overhead becomes contract cost;
- framework entry becomes market risk;
- availability uncertainty becomes individual consultant risk;
- ToR ambition becomes implementation and reputational risk;
- and deferred difficulty becomes delivery-stage risk – often landing, as ever, on whoever is least able to carry it.
If the ToR is where the polite fiction is written down, procurement is where it hardens into incentives, costs and risks. And none of this is peculiar to us, or to any one agency: the empirical research points the same way, describing a competition shaped as much by relationships and the capacity to absorb cost as by neutral merit (Sundberg, 2024).
So the question isn’t whether procurement is necessary – of course it is. It’s what happens to everything procurement leaves unresolved. Because the award doesn’t end the fiction. It gives the fiction a start date – and that is where we’ll turn next.
A note on AI from the authors. We used AI tools for some of the drafting of this post, in line with the UK Evaluation Society’s good practice guidance on the use of AI in evaluation. The creative process – the arguments, the disagreements, the sarcastic humour, and the editing – was ours throughout. Ownership of the material and responsibility for the views in it rest with us.
Our thanks to Jonathan Kuhn-Patrick and Mat Pritchard for this guest blog and for contributing their perspective to the UK Evaluation Society’s work.
Join the discussion
Comments are reviewed by a moderator before they’re published.
Reference
Sundberg, M. (2024) ‘A Quest for State Contracts: Public Procurement and the Shaping of Competitiveness in Development Consulting’, Forum for Development Studies, 51(3), pp. 363–387.
How to cite this article
Kuhn-Patrick, J. and Pritchard, M. (2026) Terms of engagement: From polite fiction to risk transfer UK Evaluation Society article. Available at: https://evaluation.org.uk/terms-of-engagement/ (Accessed: [insert date]).
Jonathan Kuhn-Patrick
Jonathan Kuhn-Patrick is an Independent Evaluation Consultant and international development and evaluation specialist with more than 30 years’ experience spanning humanitarian response, fragile states and complex programme evaluation. His career has included frontline NGO leadership, senior evaluation, policy and operational roles within DFID and FCDO, and extensive work across Africa, the Middle East and South Asia.
Now an independent consultant, Jonathan specialises in evaluability assessments, evaluation quality assurance and monitoring, evaluation and learning (MEL) framework design. His recent work has focused on fragile states, humanitarian response, climate resilience and infrastructure transparency across Asia, Africa, the Middle East and Latin America.
Jonathan led the development of the UK Evaluation Society’s good practice guidance on the use of AI in evaluation and continues to support professional learning in this area. He is also a former member of the UK Evaluation Society’s Board of Trustees.
Mat Pritchard
No comment yet, add your voice below!