Rajon Tumbokon recently joined Splunk as their Director of Engineering Enablement and Engagement. His team is responsible for the technical onboarding of engineers, technical learning, internal documentation, and knowledge sharing. Before that, he worked at LinkedIn for nine years, where he also helped create their strong learning culture.
Through experience, Rajon knows that training can be one of the most expensive business solutions. It is important to keep in mind that the goal of training is to solve business problems.
“One key thing to remember is that the primary reason your company invests in you and your team is to solve a business problem. Yes, we need to design courses or learning experiences and write documentation, but all of those are just a means to an end of adding actual business value,” said Rajon. “As your programs grow and evolve, there comes a time when your leadership questions what kind of value you’re adding to the business, and they want to know if they’re making the right investment.”
In his talk, Rajon went through some practical ways that you can use metrics to demonstrate the value and impact of training.
“I cannot tell you silver bullet metrics that’ll make you successful in your company, but what I can share is practical methods that will help you identify how you can measure so that it aligns with your company’s expectations,” Rajon said.
Four Types of Learning Measurements
There are four general ways you can measure the effectiveness of learning programs for software engineers:- Usage and adoption: Is your solution being used? Examples include training attendance, video views, and document views.
- Acceptance: Do people like your program? Examples include net promoter score (NPS) and customer satisfaction (CSAT).
- Application: Are the skills and knowledge being applied on the job? Examples include NPS after 30 days of completing training and the adoption rate of a new tool.
- Value and Impact: How does your program impact the business? Examples include the overall CSAT of training programs and the cost of internal support overhead.
Knowing What and How to Measure
While Rajon’s career has been focused on tech enablement, efficiency, and closing knowledge gaps, the same techniques to gather metrics can be applied to onboarding, collaborative learning, and peer learning initiatives. Analyze the Audience The first step is to analyze who your target audience is. You need to know how many people are in your organization, what your hiring and attrition rates are, how they are distributed around the globe, whether they are remote or not, and other factors so you can segment the audience into cohorts. This is crucial to understanding the data once you collect it. Interview Stakeholders Once you have a better understanding of the audience, the next step is to interview the stakeholders to discover what business problems need to be solved. Do they want to reduce support? Are there issues of productivity, efficiency, or accuracy? What kind of knowledge gaps do they see? Interview the Audience Even more important is interviewing different segments of your audience. Start by identifying employees from different levels, cohorts, locations, and teams to get a good cross-section and interview them. Find out what their biggest pain points are. Create a Survey You should start to notice patterns as you interview employees, which you will naturally form assumptions about. Now it’s time to validate those assumptions by sending out a survey to a larger group. Once you have the results, you can start slicing your data based on the segments you created in the audience. Define Metrics Now that you have contextual information from interviews and surveys, you can start defining metrics. Rajon suggests using the goals/signals/metrics framework, which is popular in product development and user experience:- Goals: What is the business problem being solved?
- Signals: How do we know when the goal is reached?
- Metrics: How can we quantify signals in two or more measures?