A constant challenge of Leadership and Development (L&D) teams is proving the effectiveness and overall worth of their training programs. Company leaders require hard numbers to justify investment in time-consuming and costly training. While one of the go-to training evaluation models is the Kirkpatrick Evaluation Model, its fourth level (Business Impact) can be difficult to achieve. This is where Dr. Jack Phillips comes in with his Phillips ROI Model, presented in his 1980 book Return on Investment in Training and Performance. His model is often used to measure the return on investment (ROI) of training programs and other development initiatives. Based on Kirkpatrick, it adds more detail to some levels and adds a fifth level specifically focused on the financial impact of training on a company’s bottom line, demonstrating the value and effectiveness of educational and training programs.
Overview of the Phillips ROI Model
The Phillips ROI Model evaluates training programs through five levels:
1. Reaction and Planned Action
Like Kirkpatrick’s Level 1 (Reaction), the Phillips model begins by measuring participants’ initial reactions to the training program and their intentions for applying what they learned. This often entails short questionnaires or surveys to collect data, helping assess if the required conditions for learning are present.
2. Learning
This level evaluates if learning took place, and to what extent participants acquired the intended knowledge, skills, and attitudes. This corresponds to Kirkpatrick’s Level 2 (Learning). Learners often take multiple-choice tests or quizzes before and after training.
3. Application and Implementation
This is where we start to see differences between Kirkpatrick and Phillips. Kirkpatrick’s Level 3 (Application) looks at workplace behavior to assess if participants are taking what they learned in training and using it on the job. A main criticism of Kirkpatrick is that it doesn’t gather enough data to help improve training, only acknowledges if training was put into practice. Phillips believed there was room here for improvement, expanding his Level 3 to cover both application and implementation.
How does training translate (or not) into workplace changes? Phillips addresses this question. If changes in the workplace are not occurring after training, the Phillips model helps determine if the issue is with the application or implementation of learning. While this is a subtle difference, it’s extremely helpful. For example, a factory installs new equipment and trains workers on it. The Phillips model confirms the training was successful, but its Level 3 evaluation shows the skills and knowledge are not being implemented on the shop floor. Kirkpatrick would simply tell you that training wasn’t implemented. However, Phillips indicates where the actual problem is. In this case, the employees cannot operate the new equipment because it is faulty, not because they don’t understand how to use it.
4. Business Impact
While Kirkpatrick’s Level 4 (Results) also deals with the impact of training on the business, it only focuses on results of the training on participants. Phillips is much broader, looking at the impact of training on the entire company. It helps identify if factors other than training are responsible for delivering outcomes – a major improvement over Kirkpatrick. Following Phillips, companies determine if variables such as sales data, weather, or supply problems are responsible for changes in performance metrics.
5. Return on Investment (ROI)
This is the additional fifth level that Kirkpatrick does not have. Kirkpatrick only measures training results against stakeholder expectations: Return on Expectations (ROE). The Phillips model’s fifth level measures ROI by using a cost-benefit analysis to determine the value of a training program, helping organizations understand if the money invested in training produced measurable results and what those are.
Advantages of the Phillips ROI Model
Quantify Financial Outcomes
The Phillips model can help organizations link training directly to their bottom line by evaluating participants’ reactions, learning outcomes, application, and financial returns. For example, a safety training program that reduces accidents can increase profits, and the ROI is calculated by comparing the total cost of the program to the increase in profits. Communicating a clear and quantifiable metric to stakeholders demonstrates the training’s financial value.
Identify Intangible Benefits
Phillips covers all aspects of training evaluation, from initial reaction to financial impact, offering a holistic view of the training program’s effectiveness. The model can also help organizations gauge intangible benefits resulting from training programs. These benefits include employee engagement, decreased turnover, and general workforce wellbeing.
Show How Training Supports Growth
Evaluating ROI can highlight areas where processes can be improved, leading to greater efficiency and cost savings. Phillips helps organizations understand how training programs impact business results and support company growth. Such information can assist organizations when making decisions about project prioritization and whether to continue a project or spend resources elsewhere. The focus on business impact ensures that training programs align with organizational objectives.
Identify Other Factors
Organizations can more easily identify whether factors other than training, such as seasonal demand or a new product, may be responsible for improved business results. Understanding these nuances help leaders better understand the true impact of training programs on their bottom line.
Challenges of the Phillips ROI Model
As with all evaluation models, there are some challenges. The two biggest are timing and usefulness, but there are others.
Too Late to Make Changes
One of the biggest criticisms of Phillips is that it only calculates ROI after the program is delivered, when it’s too late to make any changes to improve ROI. Some experts say it’s possible to work out a rough estimate of ROI before selecting a program. To answer this criticism, Jack and Patricia Phillips wrote The Consultant’s Guide to Results-Driven Business Proposals: How to Write Proposals That Forecast Impact and ROI in 2010, focusing exclusively on forecasting. These ROI estimates can be used when planning a learning program. Use them to determine business goals and budgets, then decide if you should revise plans before making a financial commitment.
Not Practical
Another criticism of Phillips is that knowing the ROI of a training course does not give much practical benefit to the organization. While it’s good to know, it’s not essential. If you evaluate a training program based on Phillips’ first four levels, you know if it is a success from the Level 4 metrics. After that, there’s little need to spend the time and money needed to calculate Level 5 ROI. Even Jack Phillips himself, in Patricia Phillips’ The Bottomline on ROI says that only about five to ten percent of training programs need to evaluate ROI. These are the expensive programs with the largest audiences and that use the most time and resources.
Resource Intensive
Gathering the data required for comprehensive evaluation, especially for calculating ROI, can be time-consuming and costly. This model can also be very intimidating, requiring expertise in both evaluation methodologies and financial analysis. Estimating the financial impact of training is challenging and may result in inaccurate or misleading data if not performed by knowledgeable experts. Significant changes in current processes and systems are often necessary for proper implementation of the Phillips model.
Focus on Tangible Outcomes
The emphasis on financial returns might undervalue important intangible benefits of training, such as employee satisfaction and organizational culture improvements. These are not measurable, but they are as important to company growth as results seen in hard numbers.
Final Thoughts
The Phillips ROI Model provides a comprehensive and detailed framework for evaluating the effectiveness of training programs, addressing key limitations of the Kirkpatrick Evaluation Model. By incorporating a fifth level focused on the financial impact, it allows organizations to tangibly measure the return on investment of their training initiatives. This model helps quantify not only the immediate learning outcomes and behavioral changes but also the broader business impacts, making it a valuable tool for demonstrating the financial value of training to stakeholders. Despite its challenges, the Phillips ROI Model offers a comprehensive approach that aligns training programs with organizational objectives, ensuring strategic relevance and supporting informed decision-making about training investments, ultimately developing both employee development and business growth.
Related Blogs
Mastering Kirkpatrick: Unleashing Training Excellence
Measuring Microlearning Mastery: Metrics that Matter
Is Your eLearning Accessible to Everyone?
Resources
Deller, Jonathan. “The Complete Phillips ROI Model Tutorial for Beginners.” Kodo Survey. 9/30/21. Accessed 6/13/24. https://kodosurvey.com/blog/complete-philips-roi-model-tutorial-beginners
“The 5 fundamental training evaluation models: pros, cons, and how they work.” WalkMe. 11/22/23. Accessed 6/14/24. https://change.walkme.com/training-evaluation-models/
Herrholtz, Kevin. “Extend Your Training Evaluation To Include The Phillips ROI Model.” eLearning Industry. 3/18/20. Accessed 6/13/24. https://elearningindustry.com/extend-training-evaluation-include-phillips-roi-model
“Mastering Kirkpatrick: Unleashing Training Excellence.” MATC. Accessed 6/19/24. https://www.matcgroup.com/instructional-design/mastering-kirkpatrick-unleashing-training-excellence/
Phillips, Jack J., Ph.D. and Patricia P. Phillips, Ph.D. “Measuring ROI in Executive Coaching.” ROI Institute. 2005. Accessed 6/14/24. https://roiinstitute.net/wp-content/uploads/2017/10/Measuring-ROI-in-Executive-Coaching.pdf
Tulsiani, Ravinder. “Navigating The Challenges Of Kirkpatrick’s Level 4 Evaluation In eLearning.” eLearning Industry. 11/6/23. Accessed 6/14/24. https://elearningindustry.com/navigating-the-challenges-of-kirkpatricks-level-4-evaluation-in-elearning