Here is the five-level evaluation framework that Jack developed. The graphic also shows the measurement focus and typical measures for each level.

Finally, we calculate the ROI.

As you look at the five evaluation levels and the Level 0, Input, notice the arrows leading to the next higher level. This is important because, if we fail at a level, we will also fail at the next higher level and on up. For example, if participants do not like the program, they will not learn. If they like the program but don’t learn how to change their behaviors in the learning phase, they won’t change their behaviors in the workplace. If they do not or cannot change their workplace behaviors, the impact measures will not improve. If participants do not improve the impact measures, we will end up with a negative ROI because the costs will exceed the lack of unchanged benefits.

Another important point to remember about the levels is, the higher you go, the higher the cost to evaluate the program. Therefore, you want to decide which programs you will evaluate at certain levels depending on how important the data are at that level for the program. For example, you would not evaluate a compliance program to ROI because you must do it anyway. Stop at Level 2.

After we isolate the effects of our program at Level 4, we must decide if we will convert that isolated impact measure into a tangible measure with a monetary value or leave it as an intangible measure. Intangibles measures connected to the program are reported but are not converted to a monetary value. There is a four-part test that will help you make this decision, and there is a five-step process to convert a measure to a monetary if it passes that four-part test. You will learn how to do that in Module 4 as well.

To calculate the ROI using your tangible measures, you must also make sure you have all the program costs.

Return on Investment (ROI) is a financial metric, representing the ultimate measure of program success. The Benefit-Cost Ratio (BCR) is the efficient use of funds. Both are calculations using the program benefits and costs. Learn how to do these simple calculations in this lesson.

Let’s look at the Business Alignment V model:

We have established that we want to have a worthwhile payoff. We have also established an objective for the ROI, and now we are going to determine how well we achieved that objective. We have captured all the costs, isolated impact measures, and identified the tangible benefits. Now we are going to determine if the program is worthwhile.

BCR and ROI Formula

Program benefits divided by program costs gives us our benefit cost ratio. For our return on investment calculation, we want to find out what the ROI will be, so we take the investment out which gives us our program benefits, divided by the program costs, and multiply it by a hundred to turn it into a percentage.

In this example, the question is what is the BCR and the ROI? Benefits on top and the cost on the bottom, it gives us this ratio 1.45:1. Every dollar we invest, we get $1.45 back.

To calculate the ROI, we are going to take the investment out to determine our net program benefits, divide by the cost, and multiply by a hundred to get a 45% ROI

Reporting Financial Payoff

There are different approaches to reporting financial payoff from programs. We have been primarily focusing on the cost-benefit analysis and return on investment. One of the easiest calculations is one we have not discussed, and that is the payback period. To calculate the payback period, flip the numerator and denominator numbers from the BCR and multiply that by 12. This will give you the payback period if you are reporting financial data. When you establish your target dates, your objectives, and the minimum ROI that we would like to see, you will have the criteria to determine if the program is successful. Find out what the ROI is for other initiatives, the ROI for capital expenditures, or look at what the ROI is for other types of investments. You might want to set your objective a little bit higher. Some executives may say they just want to break even. That is the ROI objective in many governments and nonprofits. Finally, whatever ROI objective the client determines, you must accept. Have this conversation upfront, during the planning phase, to define the minimum ROI that we would like to see as a result of this program. That way you have something to compare your actual ROI with to determine if your program was a success or not.

When to measure ROI

When we are looking at measuring ROI, we do not do it for every program because of the expense. We suggest that you only evaluate to ROI 5% to 10% of the time. We gave you criteria earlier to determine when it is appropriate to evaluate to ROI and when it is not. In your organization, you want to sit down and look at the levels to which you are evaluating and what percentage of our programs are going to each of these levels. Where does your evaluation team want to be a year from now, five years from now, and so on? We want to have targets so that we know how to allocate the funds in our budgets and make sure that we are evaluating the right programs to the right levels.

Rational Approach

To make sure that we have a rational approach to ROI, we must keep things simple. Using this methodology keeps things simple and understandable. The ROI Methodology is a proven methodology. Also, because we want to keep the cost down, use sampling for your calculation. Sampling will give you a good feel as to what you are getting in return for those dollars. Always account for other influencing factors. That is why we isolate the effects of our programs. If you are not isolating, then that is going to inflate your ROI and that's going to allow people to poke holes in it. They are going to undermine your credibility because they are saying that you are cooking the books, or you are trying to inflate your ROI to make the program look better when it's not true.

You want to get management involved in the process from the very beginning during the planning process. Get management involved early in the process so they will have an interest in the program, and they will want to see the data. You also want to educate the management team. One of the reasons that people resist ROI is they do not understand it. Once we educate, get comfortable, and see the benefits of it, it is much easier to get the team on board.

Be careful about how you communicate the results of your studies. We want to be careful that we do not put somebody in a corner. We always want to give people an opportunity and the benefit of the doubt. We want to position this as an opportunity to demonstrate that we are on top of things. Maybe we did not get the results that we wanted, but we are getting paid to find out what we can do better. You want to give credit where it is due, and you want to plan to do your ROI calculations from the very beginning if you choose to evaluate at the impact and ROI levels. Make sure you have the numbers and the data to be able to calculate the ROI.

Make sure that you stay very conservative in your approach when looking at the benefits and the costs. When in doubt, leave it out. When it comes to costs, when in doubt, put it in. Make sure that you are transparent in the assumptions you make, and those assumptions are covered by the ROI Methodology and the Guiding Principles. Make sure that the team knows the methodology you are using. Be careful of controversial issues, sensitive issues, and the politics within the organization. Develop relationships as you are using this methodology and getting the stakeholders involved so that you can make everyone look good. Teach others how to use the methodology too. The more people we have involved in helping, the more help we are going to get. Also, recognize that not everyone is going to buy into it. You are going to have resistance. People are going to resist change no matter what you do. When people start seeing the impact and the benefits of doing these studies, then you are going to win them over.

Be careful that you do not misuse financial terms. ROI is Return on Investment. It is not Return on Ingenuity and not Return on Imagination. ROE is return on equity. It is not return on expectations. If you start referring to ROE as return on expectations, that is a surefire way to get yourself excused from an executive board meeting.