Return on Investment
A constant challenge for learning and development professionals is to be able to demonstrate the value or return created by undertaking training. Often you will hear asked ‘so what was the return on the training investment we made?’ It’s a fair question and one that can potentially be answered by the formula:
ROI = Net Program Benefits (Benefits – Costs) x 100
Program benefits: $1,000,0000
Program cost: $ 600,000
ROI = $ 400,000 ($1,000,000-$600,000) x 100 = 66.7%
Calculating ROI faces three fundamental challenges:
- Getting accurate financial data that stands up to scrutiny from stakeholders. That’s why you will often see leadership programs include some form of business improvement project – where the project cost savings or new business generated provides the ‘top line’ of benefits
- It implies that training alone delivered the return and so isolates training as the element that delivered the change in behaviour that resulted in positive business results. It ignores managers coaching and support and the multitude of other contributing factors
- It’s retrospective, done after the training is completed and it fails to answer the questions 'What would more investment get us?’ or alternatively ‘What’s the risk by reducing the investment or by not investing at all?'
Return on Expectations
Return on Expectations (ROE) was developed by Dr Jim and Wendy Kirkpatrick about nine years ago as they created the New World Kirkpatrick Model. ROE builds on the work of learning transfer researchers such as Professor Robert Brinkerhoff, whose research has found that training alone works for only 20% of trainees in that they have an impact on business results.
So how does Return on Expectations create greater value for organisations? Fundamentally this is because ROE is designed, developed and delivered in partnership with those business stakeholders who will determine learning’s value. The Kirkpatrick Foundational Principles are the bedrock for building this partnership with the business. The Principles are:
- The end is the beginning.
- ROE is the ultimate indicator of value.
- Business partnership is necessary to bring about positive ROE.
- Value must be created before it can be demonstrated.
- A compelling chain of evidence demonstrates your bottom line value.
The end is the beginning
How will any proposed training impact on business results? Be they sales, profitability, environmental impact or patient outcomes. We are not talking about learning outcomes but how the learning will positively change the business by learners application of the skills taught bringing about a change in on the job behaviour and subsequent business results.
It’s about being clear at the start of the learners journey what the destination is; in this case the destination is impact on business results.
ROE is the ultimate indicator of value
How can this be so when we are not isolating training to calculate ROI? Stakeholders determine through extensive discussion with their learning professional(s) what their expectations are at each of the four Kirkpatrick levels starting with Results, then Behaviour, Learning and Reaction. At Level 4 Results, a common question to ask is ‘what will success look like (in terms of business results) if this training is effective?' Only once these are clear should the learning professional design a learning solution, if indeed that is the answer.
Business partnership is necessary to bring about positive ROE
How can learning professionals do this when they are often more comfortable in developing learning outcomes to build competency at a task? Fundamentally it starts with asking the questions to establish those stakeholder expectations and in doing so building the strategic bridges with executives and senior leaders in the business. It also means building tactical bridges with the managers of learners to understand the working environment, determine the critical behaviours needed on the job and to build a package of coaching, mentoring and tools and checklists that will support on the job performance post training.
Once expectations are established it is very helpful to ask what credible evidence they would like to see at each of the four levels of Results, Behaviour, Learning and Reaction. They are more likely to be interested in the Results and Behaviour than in the Learning and Reaction!
Value must be created before it can be demonstrated
How do learning professionals create real value once stakeholder expectations are established and desired business results identified? The answer is by working with managers of learners to establish the vital few critical behaviours that they need learners to be performing on the job. A helpful question for managers here is “if you were to video your team members on the job, performing to expectation, and you played back the video, describe what you would see”. This is far more effective than asking them about learning outcomes and competencies. Once you as the learning professional is clear on the critical behaviours to be performed on the job, you then need to work with the managers to develop a comprehensive package to support the transfer from learning to on the job behaviour. Crucially this also involves showing, and even training managers, on how they can support their learners and team members back on the job.
A compelling chain of evidence demonstrates your bottom line value
So what evidence is needed? This is evidence at all four levels. If you have asked your stakeholders what evidence they would like to see, it’s relatively straightforward to develop and execute a plan to collect that evidence. Bear in mind it’s not just quantitative data but also stories and other qualitative data that demonstrates value. It’s amazing the power that credible stories hold.
Fundamentally ROE is about working with business stakeholders to create value from training and to then demonstrate the degree to which those expectations have been met. ROE is a collaborative approach that creates credible value for organisations.
Download below an interview with Dr Jim Kirkpatrick about ROE.
Here are some related questions we hear asked which we will try to answer
How do you evaluate training methods?
You can measure this at two levels to determine whether the training methods themselves were effective:
- Ask your participants about how they felt about the training
- Specifically ask them if they found it favourable, if it engaged them and if it was relevant to their jobs
- Measure participants knowledge and skills as a result of the training.
- Ask your participants about how they feel using their newly acquired knowledge and skills and their commitment and confidence in using them back on the job.
These represent Kirkpatrick Level 1 (Reaction) and Kirkpatrick Level 2 (Learning)
Why is evaluating training important?
There are three key reasons to evaluate training:
- To determine the quality of the training program (training materials, facilitator, relevance to trainees) and improve it if need be
- To establish whether the training transferred to on the job behaviour
- To establish whether the training impacted positively on business results
Most evaluation work gets stuck at the first reason and limited resource is spent on reasons 2 & 3. Yet as L&D professionals we face a key challenge in demonstrating our value to the organisation, securing our place as trusted professionals in the business. Being a ‘trusted professional’ means ensuring that the learning opportunities we provide does in fact change behaviour and results in better organisational performance. L & D professionals need firstly create and then demonstrate the value of any training by partnering with key stakeholders in the business. There are many examples of organisations restructuring or eliminating the L&D function because it failed to demonstrate value to the business.
Watch this short video clip on why it’s important to evaluate training
Download this interview with Dr Jim Kirkpatrick for an understanding of why it’s important to evaluate training.