Author: Janine Ewen, Global Consultant in Public Health, Safety and Childhood Vulnerability.
In this practice-based reflection, Janine Ewen explores the ethical tensions that arise when ideal evaluation designs meet real-world constraints. Drawing on experience in the UK public sector, she considers how evaluators can make defensible trade-offs while maintaining rigour and public benefit.
Summary
In the quiet halls of theory, evaluation is a linear path towards “truth.” But for those of us working within the constraints of UK public sector budgets, shifting political cycles, and the rapid-fire demands of policy implementation, evaluation feels less like a lab experiment and more like a high-stakes negotiation. From avoiding the common pitfalls of the commissioning phase to the difficult decisions of what to cut when resources shrink, I dive into how to protect public benefit and evaluative rigour when the “perfect” method simply isn’t an option. This practice-based blog is an exploration of how we maintain our ethical compass when navigating the gap between the textbook and the field.
Rigour in the Real World: Navigating the Gap Between Idealism and Necessity
In theory, evaluation is a straight line to the ‘truth.’ In the UK public sector, however, that line is bent by budget cuts and political urgency. For the practitioner, the work feels less like a controlled experiment and more like a high-stakes negotiation—a constant balancing act between methodological ideals and the grit of the field.
As the UK Evaluation Society (UKES) continues to refine our professional standards, we must confront a difficult reality: What does integrity look like when the “perfect” method is off the table?
The Commissioning Trap: The Perils of Over-Promising
The ethics of an evaluation are often decided before the first data point is ever collected. In the rush to secure a contract or satisfy a stakeholder, there is a systemic temptation to over-promise.
The Pitfall: Proposing a gold-standard RCT (Randomised Controlled Trial) on a “bronze-standard” budget or timeline. This often ignores what Cartwright and Hardie (2012) highlight as the “it-will-work-here” factor; a rigid method that cannot be implemented properly lacks both practical and evidentiary value.
The Lesson: Professional integrity starts with the courage to say “No” to a method that cannot be delivered. True rigour is not about the complexity of the tool, but the honesty of the limitations. This embodies the “evaluative professional judgment” advocated by Schwandt (2015), where the evaluator’s role is to resist the instrumentalisation of their craft for the sake of a polished proposal.
The Practice Tip: Instead of promising “proof” of impact under tight constraints, pivot the language toward “plausible contribution” and robust “process learning.” By adopting Mayne’s (2001) framework of Contribution Analysis, we can provide meaningful insights that manage expectations and protect the credibility of the findings without claiming an unrealistic level of certainty.
Navigating the "Ethics of the Trade-off"
When budgets are cut mid-stream or political timelines move forward by six months, we face the “Ethics of the Trade-off.” This is where professional judgement—the core of our UKES standards—comes into play. How do we decide what to cut?
Don’t cut the “Who”: We often trade off sample size or longitudinal depth. However, the ethical line is crossed when we cut the voices of underserved or “hard-to-reach” groups because they are “too expensive” to recruit.
Do cut the “Breadth”: It is better to answer two research questions with high confidence than ten questions with none.
Maintaining Public Benefit Under Pressure
The UKES Guidelines for Good Practice emphasise public benefit. In the grit of the field, this means ensuring that the evaluation remains useful, even if it isn’t “perfect.”
An evaluation that is 100% rigorous but delivered six months after the policy decision has been made has a public benefit of zero. Professional judgement is the art of balancing methodological purity against decision relevance.
Key Lessons for the Field
The Challenge | The “Grit” Reality | The Ethical Response |
|---|---|---|
Data Gaps | Transparently report the gap; use triangulation (e.g., qualitative interviews) to bolster the narrative. | Planned administrative data is missing or “dirty.” |
Political Shift | Re-map the Theory of Change immediately. Document the “why” to maintain the audit trail. | The policy’s goals changed halfway through. |
Budget Cuts | A 20% reduction in funding mid-evaluation. | Prioritise the “primary” evaluation questions. Don’t thin out the quality of everything; stop doing the non-essentials. |
Conclusion: Moving from Theory to Mastery
Maintaining evaluative integrity isn’t about avoiding trade-offs; it’s about making them visible. When we are honest about what can be achieved within the “grit” of the real world, we don’t weaken the profession—we strengthen it.
We owe it to the public to provide the best possible evidence, even when the conditions are at their worst. That isn’t a failure of theory; it is the highest form of professional practice.
References
Cartwright, N. and Hardie, J. (2012) Evidence-based policy: a practical guide to doing it better. Oxford: Oxford University Press.
Mayne, J. (2001) ‘Addressing attribution through contribution analysis: using performance measures sensibly’, Canadian Journal of Program Evaluation, 16(1), pp. 1–24.
Schwandt, T.A. (2015) Evaluation foundations revisited: cultivating a life of the mind for practice. Stanford: Stanford University Press.
UK Evaluation Society (2019) Guidelines for good practice in evaluation. Available at: https://www.evaluation.org.uk (Accessed: 10 April 2026).
How to cite this article
Ewen, J. (2026) The ethics of the trade-off: Navigating the gap between ideal and real-world evaluation. UK Evaluation Society blog. Available at: https://www.evaluation.org.uk/blog-ethics-of-the-trade-off (Accessed: [insert date]).
Our thanks to Janine Ewen for this guest blog and for contributing their perspective to the UK Evaluation Society’s work.
Janine Ewen
Janine Ewen is a Global Consultant in Public Health, Safety and Childhood Vulnerability. She is a member of the Global Law Enforcement Association (GLEPHA), the Violence Against Women and Girls Research Network (VAWGRN) and hold several qualifications in public health, criminology, sociology, leadership and management. Janine runs her own Independent consultancy where she advices on needs-based work for the most vulnerable in society. Read more about Janine’s work.
