How to Measure Learning and Development

[0:00]Imagine you invested in a really expensive course to improve your presentation skills, but how do you tell if it truly made a difference? That's the same question and challenge learning and development practitioners around the world face on a daily basis. Hello and welcome to the L&D Academy. My name is Irina and today let's explore some of the most fundamental things you need to know about measuring learning and development. First of all, what is learning evaluation? We can describe it as the total worth or value of L&D in monetary and non-monetary terms. So, really there are three key elements here: worth or value, monetary and non-monetary. Let's try to make sense of these. What exactly is the worth or value of L&D? Well, we provide trainings and coaching, mentoring, on-the-job learning, online courses and many other learning experiences. But why is that important? Well, you can say that we help people develop and therefore the organization develops through its people. And that makes a lot of sense, but let's be honest. No chief financial officer will ever be satisfied with such an answer. What they need to know is what is our worth in numbers? And so we arrive at the other two key elements: monetary and non-monetary value. In the monetary metrics, you could look for things like the L&D budget and spending, how many people attended a learning event and how many didn't show up. How many new learning programs were launched in that year? How many training hours each employee had? What was the return on investment? What's the cost of training versus not training? Reduction in turnover costs, increased sales or production output, reduction in waste or error rates and so on. In the non-monetary column, you could put things like how satisfied were learners after each intervention? What was the employee engagement with various L&D activities? How much time does it take to master a competence? How was performance improved as a result of the learning? How confident learners were before and after the learning event? How many internal promotions happen in the set period of time? What was the employee's turnover rate? How learning events have influenced the organizational culture and so on. And this is where we arrive at the second question, what to measure? Now, there are several authors and popular models out there, like Kirk Patrick's for levels and Phillips ROI. But at the heart of it, there are seven things you want to measure. Reactions, knowledge, skills, attitudes, application, results and return on investment. In other words, did participants like the learning event? What knowledge did they gain and retain and what skills did they gain and retain? How are they behaving differently as a result of the learning event? Are participants applying what they have learned back on the job? What is the effect of the learning event on the business bottom line and what is the monetary return on investment of the learning experience? And that takes us nicely to the next question. How to measure it? At your disposal, you have several research methods and data collection options. So let's break it down for each of the seven metrics. For reactions, you need to approach your learners pretty much as soon as the event ends. The most common methods used are the satisfaction surveys, which are also known in the industry as happy sheets. To assess the knowledge, you can use pre and post-course tests or quizzes. Timing here is essential as we can assess before, during and after the learning event to see the full picture of knowledge acquisition. To understand what skills our learners have acquired, you need to get a bit creative. You can use things like role plays, case studies, and exercises during the learning event. However, you can also give it some time and then talk to the participants peers and managers to see what changes in skills they have observed. To measure the changes in attitude, first, you need to wait some time. Attitude doesn't change with a snap of a finger. I would recommend waiting anywhere between three to six months after the learning event and then you can use things like attitude surveys, like a 360-degree feedback. You can also interview participants and their supervisors and observe them back on the job and also review their performance appraisals. To see how much of the learning material has been applied back on the job again, you need to wait anywhere between six to twelve months. Some of the more common ways to measure this is by reviewing the action plans created during the learning event, post-event interview of participants and their supervisors, and even a good old-fashioned questionnaire. To measure the results on the bottom line of the business, you once again need to give it some time. Six to twelve months is probably a good period of time. For this metric, you need to get even more analytical. You can do a cost-benefit analysis, you can track operational and performance data, employee retention, sales revenues, engagement surveys, number of customer complaints, customer service ratings, operating costs, product defects, and so on. Usually, you'll agree on what you will measure at the beginning of the learning design process. In other words, if I'm creating a presentation skills workshop, before I design the session, I will have agreed with my stakeholders, what will be measured at the end. It will be things like higher rate of successful project pitches, measurable improvement of slide designs, customer feedback on presentations and so on. Finally, the most elusive metric, the return on investment. Now, we did a whole video on measuring the learning ROI, which you can check out here on the screen or in the description down below. Without going into too much details here, the most common tools to measure the ROI are the cost benefit analysis, trend lines, participants and supervisor's estimates, and of course the ROI formula, which is net monetary benefits, that is benefit minus cost divided by the total cost and multiplied by a hundred. It's also worth mentioning that there are some challenges of measuring L&D. If I speak Spanish fluently, is it because of my teacher, the curriculum, the classmates who help me, or was it because I lived in Spain for a year where I had to speak Spanish? By the way, I don't speak Spanish, don't test me on this one. But this is essentially the biggest challenge of measuring L&D. There are many different factors of affecting someone's performance. How can you isolate the ones that are directly affected by learning and those that aren't? There isn't a simple way to solve this. As an L&D professional, you must employ a nuanced approach, combining quantitative metrics with qualitative insights to really capture the holistic impact of learning experiences. This involves not just looking at test scores or completion rates, but also gathering feedback, observing behavioral changes, considering external factors that can also contribute to performance. If you want to learn more about this topic, we have a short mini online course on learning evaluation that delves deeper into the evaluation models and processes. And exclusively for our YouTube viewers, you can use a 10% discount by using the code in the description. Hope to see you there.