Institutions invest heavily in workshops, courses and certification. When budgets are reviewed, the evidence offered is often an attendance list and an average satisfaction score.
Neither answers the question a board, funder or ministry eventually asks: did the way people work change, and did that change make a difference? This guide sets out the most widely used framework for evaluating training, what the research says about each kind of evidence, and five steps for building evaluation into a programme from the start.
The four levels most evaluators use
The best-known framework was developed by Donald Kirkpatrick and is now maintained by Kirkpatrick Partners, which says it has been in use for “more than seven decades”. It sorts evidence about a programme into four levels.
Level 1: Reaction
Whether participants found the training favourable, engaging and relevant. The current guidance puts relevance ahead of enjoyment: if a programme is not relevant to participants, it says, application “will most certainly not occur”.
Level 2: Learning
Whether participants gained the intended knowledge, skills, attitudes, confidence and commitment. Pre- and post-tests are the traditional measure, but role plays and “teach-backs”, where participants explain a task to others, can suit practical content better.
Level 3: Behaviour
Whether participants apply what they learned once back at work, and whether they are supported and held accountable for doing so. This is where training either transfers into practice or fades.
Level 4: Results
Whether the organisational outcomes the programme targeted actually occur. Kirkpatrick Partners now advises starting the planning here, with the purpose of the programme, and working backwards. It also notes that the levels are “linked, yet the model is not linear”.
What the research says about satisfaction scores
Reaction forms are the easiest evidence to collect, and by far the most common. A 2003 meta-analysis in the Journal of Applied Psychology by Winfred Arthur Jr and colleagues cited a 2002 industry benchmarking survey in which 78% of organisations used reaction measures, compared with 32% for learning and 7% for results.
The difficulty is that enjoying a course is a weak guide to learning from it. A meta-analysis of 34 studies by George Alliger and colleagues, published in Personnel Psychology in 1997, found an average correlation of just 0.02 between “affective” reactions (how much participants liked the training) and immediate learning.
“Utility” reactions, about how useful participants judged the training, did better: a correlation of 0.26 with immediate learning and 0.18 with on-the-job transfer. Even so, the authors concluded that “reaction measures cannot be used as surrogates of other measures”.
Later reviews point the same way. A 2022 evidence summary by the Quality Improvement Center for Workforce Development (QIC-WD) reports that a 2008 meta-analysis by Traci Sitzmann and colleagues found reactions “only modestly related to immediate knowledge outcomes and unrelated to delayed knowledge”. It also reports that a 2010 meta-analysis of training transfer by Brian Blume and colleagues found a modest link for utility reactions and none for affective reactions.
That does not make feedback forms worthless. The same summary notes that reactions are strongly connected to motivation after training, and Kirkpatrick Partners recommends short “pulse checks” during a programme so that problems can be fixed while it is still running. The point is narrower: a high satisfaction score is not, on its own, evidence of impact.
Training does work, when it is measured
The counter-evidence matters as much. Arthur and colleagues’ 2003 meta-analysis, drawing on 397 effect sizes from 162 sources, found “a medium to large effect size for organizational training”, with average effects between 0.60 and 0.63 standard deviations across reaction, learning, behaviour and results measures.
Training, in other words, generally does change knowledge, behaviour and results. The gap is in what gets measured. In the same study, learning measures made up 59% of the effect sizes and behaviour 31%, while results accounted for only 7%.
There are also limits to this evidence. Most of it comes from organisations in high-income countries, much of it is decades old, and the Alliger analysis rests on 34 studies. Results-level evaluation is rare precisely because outcomes take time to appear and have many causes besides training.
Five steps to plan an evaluation that answers the real question
1. Start with the result, not the course
Agree with leaders which outcome the training is meant to improve, such as fewer errors in surveillance reports, faster grant processing or safer infection-prevention practice. Then find the indicators that already track it; Kirkpatrick Partners observes that most are “already tracked somewhere within the organization”. A theory of change helps set out how training is expected to lead to that result.
2. Plan the evaluation before delivery
The UK Treasury’s Magenta Book, the government’s central guidance on evaluation, updated in May 2026, states that evaluation “should be built into an intervention’s design and delivery from the earliest stages”. For training, that means collecting baseline data before the first session rather than reconstructing it afterwards.
3. Ask useful questions at Level 1
Replace “did you enjoy the session?” with questions about relevance and intention: which task will you use this for, and what might stop you? The research above suggests that utility-type questions are better predictors of transfer, and the answers flag barriers early.
4. Measure behaviour at work, early and more than once
Observation, work samples and supervisor check-ins show whether practice has changed. Kirkpatrick Partners warns against waiting too long: measuring behaviour “cannot begin at 90 days after learning”, because by then the chance to correct course has gone. Design choices that support application are covered in From training to capability.
5. Match the method to the stakes
Not every webinar needs a controlled study, but a large or high-risk programme may justify comparison groups and mixed methods. The World Health Organization’s 2024 evaluation methodology guide for its environmental cleaning training package, for example, combines process and impact evaluation using “qualitative, quantitative, and microbiological” methods. WHO’s Evaluation Practice Handbook (2013) covers how to prepare, conduct and follow up evaluations more generally.
Common pitfalls
- Treating attendance or certificates as outcomes. They show that training happened, not what it achieved.
- Measuring once, immediately after the session. Learning that is not used decays; follow-up is where transfer becomes visible.
- Ignoring the work environment. Kirkpatrick Partners describes the “performance environment” as able to support or block progress at every level. If managers leave no room for new practice, even strong training will show little change.
- Over-claiming results. When an outcome improves, other changes may explain it. Evaluations are more credible when they say so.
Why it matters for institutions
For ministries, health systems and development programmes, training is often the largest single investment in people. Evaluating it well turns that spending into evidence: what worked, for whom, and what support practice needs afterwards.
Praxis Global Institute designs its Education & Training programmes, including the IPR Program, around application in practice, and institutions planning how to evaluate their own learning programmes can contact the team.
Praxis Editorial Team
The Praxis Editorial Team publishes Praxis Global Institute’s explainers, practical guides and analysis of new evidence and policy in global health, education and professional capability, ethical AI and technology, research and data, and institutional strengthening. Every article is researched from primary sources, cites its evidence and is published under the editorial direction of its editors.
Facing a similar challenge in your institution?
We work with teams on research, data, digital systems, and learning that hold up in practice.
