In this article, Jonathan Kuhn-Patrick, UK Evaluation Society trustee and AI lead, talks about his AI journey and co-developing our new cross-sector AI guidance.
Like many life – changing encounters, mine with artificial intelligence began in a distinctly mundane setting: a conference corridor.
It was 2023, at the UK Evaluation Society conference, and I had just emerged from Rick Davies’ presentation on AI with the glazed expression of someone who had glimpsed the future but wasn’t entirely sure what to do with the information.
Rick—whose reputation in evaluation circles hardly needs embellishment—had demonstrated how he used AI in his evaluation work, carefully balancing breathless possibilities with sobering limitations.
While ChatGPT and its digital siblings had been making headlines for months, I had remained largely on the sidelines—a professional evaluator who, rather ironically, had failed to evaluate whether this new technology might be worth my attention. That oversight was about to be remedied with the efficiency that only hindsight allows us to appreciate.
Standing in that corridor, still processing what I’d witnessed, I did what any rationally curious person would do: I immediately downloaded Claude onto my phone. Rick had mentioned using Claude as his preferred tool, and if it was good enough for someone who clearly knew what he was talking about, it seemed a reasonable starting point for someone who manifestly did not.
My initial forays were tentative affairs—general queries that I might previously have directed at Google. Yet I quickly discovered that Claude’s responses possessed a depth and comprehensiveness that made my previous search strategies seem rather pedestrian. What struck me wasn’t merely the speed of responses (though the real – time interaction was remarkable), but the sheer breadth of knowledge it appeared to draw upon. It was like engaging with a superintelligent being who happened to be available for conversation at any hour, possessed infinite patience, and never seemed to tire of my questions—though, as I would later discover, these very qualities would become sources of both fascination and concern.
The moment of genuine revelation came when I recognised AI’s transformative potential for evaluation work. Here was a tool that could process vast quantities of data and analyse them with a speed that made my traditional approaches seem almost quaint. For someone who had spent years wading through documents and synthesising findings at merely human velocity, this was nothing short of revolutionary.
My enthusiasm, I must confess, knew few bounds. I found myself returning home each evening with fresh tales of AI’s capabilities, regaling my increasingly patient wife with elaborate accounts of what this technology might accomplish. Her tolerance for my repeated digital evangelism began showing signs of strain. While I marvelled at AI’s possibilities, she maintained a more circumspect perspective, focusing on potential downsides with the sort of level – headed scepticism that probably explains why she married an evaluator in the first place.
My first professional deployment came during an evaluability assessment, where I tasked AI with analysing substantial volumes of material. Being committed to transparency – even absent established protocols for disclosing AI use – I was open about how the technology had informed my analysis. The results exceeded expectations: it not only processed information with remarkable efficiency but identified insights about comparable programmes that I might never have discovered through conventional means. I had, rather unwittingly, stumbled upon what would become a recurring theme – AI’s capacity to reveal connections and patterns that human analysis might miss, not through superior intelligence necessarily, but through its ability to hold vast amounts of information simultaneously in view.
This newfound efficiency came with delicious ironies. As my wife astutely observed, saving time when paid by the hour wasn’t necessarily the shrewdest economic strategy. But the time saved allowed me to delve deeper into analysis, to pursue lines of inquiry that might otherwise have been sacrificed to deadline pressures. It was my first taste of what I would come to understand as AI’s greatest gift: not replacing human thinking but amplifying it.
Isaac Newton famously observed that his scientific achievements came from “standing on the shoulders of giants” – building upon the work of those who came before him. In my relationship with AI, I’ve found myself in a curious inversion of this metaphor: here I am, standing on the shoulders of a digital giant whose computational strength allows me to see patterns and connections that might otherwise remain beyond my reach. Yet the seeing, the interpreting, the meaning – making these remain fundamentally human endeavours. AI provides the platform; I provide the perspective. And I take full responsibility for whatever I produce with the assistance of AI.
This experience opened my eyes to AI’s broader potential within evaluation practice. The technology excels at processing vast quantities of data—analysing interview transcripts, synthesising literature reviews, identifying patterns across multiple sources, and conducting rapid environmental scans to locate comparable programmes. It can assist with qualitative coding, support statistical analysis, and help structure complex arguments for different audiences. AI proves particularly valuable in evaluation’s research phases: stakeholder mapping, policy context analysis, and the kind of comprehensive literature synthesis that might otherwise consume weeks of human effort.
“It was my first taste of what I would come to understand as AI’s greatest gift: not replacing human thinking but amplifying it.”
Yet clear boundaries define this utility. AI should never be entrusted with making value judgements about programme merit or worth – that remains the evaluator’s fundamental responsibility. Final evaluation recommendations require human professional judgement, as does the delicate work of stakeholder engagement and relationship management. Methodological decisions, particularly those involving trade – offs between competing evaluation approaches, demand the kind of contextual wisdom that emerges from experience rather than computation. The handling of sensitive or confidential data requires careful human oversight and adherence to protocols that AI cannot navigate independently.
These limitations reflect current capabilities rather than immutable truths. Given AI’s rapid development, I reserve the right to revisit these boundary statements in the not-too-distant future – though I suspect the core principle of human judgement in evaluation will prove more durable than the specific tasks that require it.
Yet this initial euphoria was tempered by healthy paranoia about accuracy. I spent considerable time fact-checking AI’s work, hunting for the hallucinations and errors I’d heard could lurk within seemingly authoritative responses. There were indeed mistakes – enough to vindicate my caution, but few enough to confirm the technology’s essential utility. It became a delicate dance of trust and verification that would characterise all my subsequent AI interactions.
What I lacked was systematic understanding of how to navigate this partnership between human judgement and artificial capability. I was operating on instinct and enthusiasm – a state that, while energising, hardly constituted sustainable methodology. The framework for transforming this relationship from haphazard collaboration into deliberate amplification would come through Anthropic’s AI Fluency course.
The AI Fluency framework proved revelatory in its systematic approach to understanding not just AI’s capabilities, but its appropriate applications. It taught me to think in terms of task delegation – distinguishing between work that benefits from AI’s computational strengths (pattern recognition, data synthesis, rapid iteration) and work requiring distinctly human capabilities (contextual judgement, ethical reasoning, creative synthesis). The framework helped me understand that effective AI use isn’t about finding tasks that AI can do, but about identifying tasks where AI’s particular strengths complement human limitations, and vice versa.
Perhaps most crucially, it provided structure for risk assessment—teaching me to recognise when AI outputs required rigorous verification versus when they could be trusted with greater confidence. This wasn’t about blanket scepticism, but about developing what might be called “calibrated trust” – understanding the conditions under which AI performs reliably and those where human oversight becomes essential.
This framework would provide the conceptual scaffolding to help me understand not just what AI could do, but what I should ask it to do, and perhaps more importantly, what I should never delegate to it at all. But that transformation from enthusiastic experimenter to systematic practitioner—well, that’s another section entirely.
And yes, before you ask: I did use AI to help write this blog. The irony is not lost on me.
Author bio – Jonathan Kuhn-Patrick
Jonathan Kuhn-Patrick is Treasurer of the UK Evaluation Society. He is a senior evaluation specialist with over 30 years of experience designing and leading evaluations across fragile states and humanitarian contexts.
Jonathan served as Evaluation Adviser in DFID’s Evaluation Department, where he helped establish evaluation approaches for humanitarian and conflict programmes now used across government departments. His evaluation work focuses on designing MEL frameworks for large-scale humanitarian responses and conducting evaluability assessments for complex development portfolios.
His evaluation expertise spans high-risk environments across South Asia, the Middle East and Africa, developing methodologies adapted to conflict-affected contexts. Jonathan brings extensive experience in financial oversight of evaluation budgets and complex organisations to his role as Treasurer, ensuring the Society’s resources effectively support the UK’s evaluation community.
