The Phillips ROI Model: An Analysis

A constant challenge of Leadership and Development (L&D) teams is proving the effectiveness and overall worth of their training programs. Company leaders require hard numbers to justify investment in time-consuming and costly training. While one of the go-to training evaluation models is the Kirkpatrick Evaluation Model, its fourth level (Business Impact) can be difficult to achieve. This is where Dr. Jack Phillips comes in with his Phillips ROI Model, presented in his 1980 book Return on Investment in Training and Performance. His model is often used to measure the return on investment (ROI) of training programs and other development initiatives. Based on Kirkpatrick, it adds more detail to some levels and adds a fifth level specifically focused on the financial impact of training on a company’s bottom line, demonstrating the value and effectiveness of educational and training programs.

Three increasingly larger stacks of coins with small plants "growing" out of them. Text below image reads: "If you want to show how your programs are truly helping the company save money and tangibly increase profits, the Philips model is the solution.” Kevin Herrholtz Vice President of Client Success, AddShoppers "

Overview of the Phillips ROI Model

The Phillips ROI Model evaluates training programs through five levels:

1. Reaction and Planned Action

Like Kirkpatrick’s Level 1 (Reaction), the Phillips model begins by measuring participants’ initial reactions to the training program and their intentions for applying what they learned. This often entails short questionnaires or surveys to collect data, helping assess if the required conditions for learning are present.

2. Learning

This level evaluates if learning took place, and to what extent participants acquired the intended knowledge, skills, and attitudes. This corresponds to Kirkpatrick’s Level 2 (Learning). Learners often take multiple-choice tests or quizzes before and after training.

3. Application and Implementation

This is where we start to see differences between Kirkpatrick and Phillips. Kirkpatrick’s Level 3 (Application) looks at workplace behavior to assess if participants are taking what they learned in training and using it on the job. A main criticism of Kirkpatrick is that it doesn’t gather enough data to help improve training, only acknowledges if training was put into practice. Phillips believed there was room here for improvement, expanding his Level 3 to cover both application and implementation.

How does training translate (or not) into workplace changes? Phillips addresses this question. If changes in the workplace are not occurring after training, the Phillips model helps determine if the issue is with the application or implementation of learning. While this is a subtle difference, it’s extremely helpful. For example, a factory installs new equipment and trains workers on it. The Phillips model confirms the training was successful, but its Level 3 evaluation shows the skills and knowledge are not being implemented on the shop floor. Kirkpatrick would simply tell you that training wasn’t implemented. However, Phillips indicates where the actual problem is. In this case, the employees cannot operate the new equipment because it is faulty, not because they don’t understand how to use it.

4. Business Impact

While Kirkpatrick’s Level 4 (Results) also deals with the impact of training on the business, it only focuses on results of the training on participants. Phillips is much broader, looking at the impact of training on the entire company. It helps identify if factors other than training are responsible for delivering outcomes – a major improvement over Kirkpatrick. Following Phillips, companies determine if variables such as sales data, weather, or supply problems are responsible for changes in performance metrics.

5. Return on Investment (ROI)

This is the additional fifth level that Kirkpatrick does not have. Kirkpatrick only measures training results against stakeholder expectations: Return on Expectations (ROE). The Phillips model’s fifth level measures ROI by using a cost-benefit analysis to determine the value of a training program, helping organizations understand if the money invested in training produced measurable results and what those are.

Person looking at tablet showing various graphs and charts. Text under image reads: "The concept of ROI has been used for over 300 years as a business tool. It is not a new fad passing through the organization. Instead, it is an old, familiar friend that is now used in unfamiliar places such as human capital, quality and technology." Jack J. Phillips, Ph.D. and Patricia P. Phillips, Ph.D. Measuring ROI in Executive Coaching"

Advantages of the Phillips ROI Model

The advantages to using the Phillips model are numerous.

Quantify Financial Outcomes

The Phillips model can help organizations link training directly to their bottom line by evaluating participants’ reactions, learning outcomes, application, and financial returns. For example, a safety training program that reduces accidents can increase profits, and the ROI is calculated by comparing the total cost of the program to the increase in profits. Communicating a clear and quantifiable metric to stakeholders demonstrates the training’s financial value.

Identify Intangible Benefits

Phillips covers all aspects of training evaluation, from initial reaction to financial impact, offering a holistic view of the training program’s effectiveness. The model can also help organizations gauge intangible benefits resulting from training programs. These benefits include employee engagement, decreased turnover, and general workforce wellbeing.

Show How Training Supports Growth

Evaluating ROI can highlight areas where processes can be improved, leading to greater efficiency and cost savings. Phillips helps organizations understand how training programs impact business results and support company growth. Such information can assist organizations when making decisions about project prioritization and whether to continue a project or spend resources elsewhere. The focus on business impact ensures that training programs align with organizational objectives.

Identify Other Factors

Organizations can more easily identify whether factors other than training, such as seasonal demand or a new product, may be responsible for improved business results. Understanding these nuances help leaders better understand the true impact of training programs on their bottom line.

Man concentrating on game of chess using light and medium toned wooden chess pieces. Text under image reads: "The challenge of isolating the impact of training is often compounded by the multifarious nature of organizational ecosystems. Various internal and external factors, each with its unique influence, create a complex tapestry that obscures the direct attribution of performance improvements to training." Ravinder Tulsiani Chief Learning Architect & Strategist eLearning Industry"

Challenges of the Phillips ROI Model

As with all evaluation models, there are some challenges. The two biggest are timing and usefulness, but there are others.

Too Late to Make Changes

One of the biggest criticisms of Phillips is that it only calculates ROI after the program is delivered, when it’s too late to make any changes to improve ROI. Some experts say it’s possible to work out a rough estimate of ROI before selecting a program. To answer this criticism, Jack and Patricia Phillips wrote The Consultant’s Guide to Results-Driven Business Proposals: How to Write Proposals That Forecast Impact and ROI in 2010, focusing exclusively on forecasting. These ROI estimates can be used when planning a learning program. Use them to determine business goals and budgets, then decide if you should revise plans before making a financial commitment.

Not Practical

Another criticism of Phillips is that knowing the ROI of a training course does not give much practical benefit to the organization. While it’s good to know, it’s not essential. If you evaluate a training program based on Phillips’ first four levels, you know if it is a success from the Level 4 metrics. After that, there’s little need to spend the time and money needed to calculate Level 5 ROI. Even Jack Phillips himself, in Patricia Phillips’ The Bottomline on ROI says that only about five to ten percent of training programs need to evaluate ROI. These are the expensive programs with the largest audiences and that use the most time and resources.

Resource Intensive

Gathering the data required for comprehensive evaluation, especially for calculating ROI, can be time-consuming and costly. This model can also be very intimidating, requiring expertise in both evaluation methodologies and financial analysis. Estimating the financial impact of training is challenging and may result in inaccurate or misleading data if not performed by knowledgeable experts. Significant changes in current processes and systems are often necessary for proper implementation of the Phillips model.

Focus on Tangible Outcomes

The emphasis on financial returns might undervalue important intangible benefits of training, such as employee satisfaction and organizational culture improvements. These are not measurable, but they are as important to company growth as results seen in hard numbers.

Smiling person sitting at laptop with hands up in celebration. Text under image reads: "By evaluating participants' reactions, learning outcomes, application, and ultimately, the financial returns, this model offers a comprehensive perspective on training impact, making it a valuable choice for stakeholders looking for a clear ROI.” WalkMe Team Change Management Blog"

Final Thoughts

The Phillips ROI Model provides a comprehensive and detailed framework for evaluating the effectiveness of training programs, addressing key limitations of the Kirkpatrick Evaluation Model. By incorporating a fifth level focused on the financial impact, it allows organizations to tangibly measure the return on investment of their training initiatives. This model helps quantify not only the immediate learning outcomes and behavioral changes but also the broader business impacts, making it a valuable tool for demonstrating the financial value of training to stakeholders. Despite its challenges, the Phillips ROI Model offers a comprehensive approach that aligns training programs with organizational objectives, ensuring strategic relevance and supporting informed decision-making about training investments, ultimately developing both employee development and business growth.


Related Blogs

Mastering Kirkpatrick: Unleashing Training Excellence  

Measuring Microlearning Mastery: Metrics that Matter

Is Your eLearning Accessible to Everyone?

 

Resources

Deller, Jonathan. “The Complete Phillips ROI Model Tutorial for Beginners.” Kodo Survey. 9/30/21. Accessed 6/13/24. https://kodosurvey.com/blog/complete-philips-roi-model-tutorial-beginners

“The 5 fundamental training evaluation models: pros, cons, and how they work.” WalkMe. 11/22/23. Accessed 6/14/24. https://change.walkme.com/training-evaluation-models/

Herrholtz, Kevin. “Extend Your Training Evaluation To Include The Phillips ROI Model.” eLearning Industry. 3/18/20. Accessed 6/13/24. https://elearningindustry.com/extend-training-evaluation-include-phillips-roi-model

“Mastering Kirkpatrick: Unleashing Training Excellence.” MATC. Accessed 6/19/24. https://www.matcgroup.com/instructional-design/mastering-kirkpatrick-unleashing-training-excellence/

Phillips, Jack J., Ph.D. and Patricia P. Phillips, Ph.D. “Measuring ROI in Executive Coaching.” ROI Institute. 2005. Accessed 6/14/24. https://roiinstitute.net/wp-content/uploads/2017/10/Measuring-ROI-in-Executive-Coaching.pdf

Tulsiani, Ravinder. “Navigating The Challenges Of Kirkpatrick’s Level 4 Evaluation In eLearning.” eLearning Industry. 11/6/23. Accessed 6/14/24. https://elearningindustry.com/navigating-the-challenges-of-kirkpatricks-level-4-evaluation-in-elearning