Skip to main content
All CollectionsLEARNROI METHODOLOGY
Why the ROI Methodology?
Why the ROI Methodology?

The ROI Methodology in 12 Easy Steps

Asia Ali avatar
Written by Asia Ali
Updated over 3 months ago

For a full breakdown, examples, and step-by-step guide for implementing the ROI Methodology, check out this Application Guide!

A guide for developing Impact and ROI studies for programs, projects, and solutions in the following areas:

  • Human Resources/Human Capital

  • Training/Learning/Development

  • Leadership/Coaching/Mentoring

  • Knowledge Management/Transfer

  • Recognition/Incentives/Engagement

  • Work Arrangement Systems

  • Change Management/Culture

  • Talent Management/Retention

  • Policies/Procedures/Processes

  • Technology/Systems/IT

  • Meetings/Events/Conferences

  • Marketing/Advertisement/Promotions

  • Compliance/Risk Management

  • Organization Development/Consulting

  • Project Management Solutions

  • Quality/Six Sigma/Lean Engineering

  • Communications/Public Relations

  • Public Policy/Social Programs

  • Creativity/Innovation

  • Ethics/Integrity

  • Safety/Health/Fitness Programs

  • Environment/Sustainability

  • Healthcare Initiatives

  • Schools/Colleges/Universities

  • Public Sector/Nonprofits

  • Faith-Based Programs

The ROI Methodology® is a balanced approach to measurement and evaluation that generates six types of data:

  • Reaction and Planned Action

  • Learning

  • Application and Implementation

  • Impact

  • Return on Investment

  • Intangibles

The process includes a step to isolate the effects of the project, program, or solution.

Six Levels of Data

Level 0 - Input

Measurement Focus:

  • Input into programs, including indicators representing scope, volumes, times, costs, and efficiencies

Typical Measures:

  • Types of programs

  • Number of programs

  • Number of people involved

  • Hours of involvement

  • Costs

Level 1 - Reaction and Planned Action

Measurement Focus:

  • Reaction to the programs, including participants’ perceived value and planned action to make them successful

Typical Measures:

  • Relevance

  • Importance

  • Usefulness

  • Appropriateness

  • Intent to use

  • Motivational

  • Recommended to others

Level 2 - Learning

Measurement Focus:

  • Reaction to the programs, including participants’ perceived value and planned action to make them successful

Typical Measures:

  • Skills

  • Learning

  • Knowledge

  • Capacity

  • Competencies

  • Confidences

  • Contacts

Level 3 - Application and Implementation

Measurement Focus:

  • Application and use of knowledge, skills, and competencies, including progress made and implementation success

Typical Measures:

  • Behaviors used

  • Extent of use

  • Task completion

  • Frequency of use

  • Actions completed

  • Success with use

  • Barriers to use

  • Enablers to use

  • Engagement

Level 4 - Impact

Measurement Focus:

  • The impact of implementing programs and processes expressed as improvement in business measures directly linked to the program or project

Typical Measures:

  • Graduation rates

  • Infant mortality

  • Crime rates

  • Productivity

  • Revenue

  • Quality

  • Jobs created

  • Efficiency

  • Incidents of disease

  • Retention

  • Customer satisfaction

Level 5 - ROI

Measurement Focus:

  • Comparison of monetary benefits from the program to program costs

Typical Measures:

  • Benefit-Cost Ratio (BCR)

  • ROI (%)

  • Payback period

Step 1

Start with Why: Align Programs with the Business

Starting with “why” is the first step in the ROI Methodology. The “why” of programs is the business need.

Payoff Needs

At one extreme, some organizations do not pursue new programs unless there is a direct business connection. A more practical approach is to be selective, making the connection when the request seems to be expensive, critical to the organization, part of a strategy, or important to the management team. The first issue is to address the potential payoff needs.

The analysis can be simple or comprehensive. A program’s ultimate payoff will be in the form of profit, cost savings, or cost avoidance.

Business Needs

The second issue is pinpointing one or more business measures already in the system that need to improve as a result of the program. Determining specific business needs is directly linked to developing the potential payoff. When determining the business needs, specific measures are pinpointed in an effort to clearly assess the business situation.

Step 2

Make it Feasible: Select the Right Solution

With business needs in hand, the next step is determining how to improve the business measures. This step identifies the causes of problems or explores the various approaches to address an opportunity.

Performance Needs

Some program implementers are moving from request taker to business contributor. This may require a new role and new skills for the team. These individuals are resisting the temptation to say yes to every request for a new program. Instead, they describe the problem or opportunity with business impact measures and identify the solution or solutions that will influence these business needs.

Learning Needs

Performance needs uncovered in the previous step often require a learning component to ensure all parties know what they need to do and how to do it as the performance is delivered. In some cases, learning becomes the principal solution, as in competency development, major technology changes, and system installations. For other programs, learning is a minor part of the solution and often involves simply understanding the solution, such as the process, procedure, or policy. For example, when implementing a new ethics policy for an organization, the learning component requires understanding how the policy works and the participant’s role in it. In short, a learning solution is not always needed, but all solutions have a learning component.

Preference Needs

The final level of needs analysis is based on preferences, which drive the program requirements. Essentially, individuals prefer certain content, processes, schedules, or activities for the structure of the program. These preferences define how the particular program will be implemented. If the program is a solution to a problem or taking advantage of an opportunity, this step defines how the solution will be implemented and how participants should perceive its value.

Step 3

Expect Success: Plan for Results

Several issues are involved in expecting success: define success for the program, set objectives at multiple levels, define responsibilities of all stakeholders to achieve success, complete the data collection plan, and complete the ROI analysis plan.

Developing Objectives

Typical Reaction Objectives

At the end of the program, participants should rate each of the following statements at least a 4 out of 5 on a 5-point scale:

  • The program is relevant to the needs of the target audience.

  • The facilitators/organizers responded to my questions clearly.

  • The program is valuable to this mission, cause, or organization.

  • The program is important to my (our) success.

  • The program is motivational for me (us).

  • The program is practical.

  • The program contained new information.

  • The program represented a good use of my time.

  • I will recommend the program to others.

  • I will use the concepts and materials from this program.

Learning objectives can have three components:

  • Performance—what the participant or stakeholder will be able to do as a result of the program

  • Conditions under which the participant or stakeholder will perform the various tasks and processes

  • Criteria—the degree or level of proficiency necessary to perform a new task, process, or procedure that is part of the solution

Data Collection Plan

Data collection planning answers fundamental questions about data collection: What, How, Who, When, Where, and How Much?

ROI Analysis Plan

The ROI Analysis Plan details how improvement in business measures will be isolated to the program and converted to monetary value. Cost categories, intangible benefits, and target audiences for communication are also identified.

Evaluation Project Plan

The project plan details each step of the evaluation.

Step 4

Make it Matter: Design for Input, Reaction, and Learning

Make it matter is a critical concept for program input (who’s involved), reaction (how participants perceive it), and learning (what participants will learn).

Data Collection

Data are captured through a variety of measurement processes ranging from formal testing to informal self-assessments. Several methods are used, including:

  • Surveys and questionnaires—determine the extent to which participants have acquired skills, knowledge, and information

  • Facilitation assessments—ratings from facilitators or project leaders based on observations during the project

  • Written tests and exercises—measure changes in knowledge and skills

  • Skill practices—help assess the degree of applied learning and acquisition of problem-solving skills

  • Performance demonstrations—provide direct evaluation of the ability to apply knowledge and skills

  • Simulations—enable assessment of skills and knowledge acquisition

  • Team assessments—assess the extent of skills and knowledge acquisition

  • Skill/confidence building exercises—an interactive approach to capturing skill and knowledge levels

Step 5

Make it Stick: Design for Application and Impact

Make it stick focuses on two types of data that are collected after a program is implemented: (Level 3) Application and (Level 4) Impact.

Application and Impact Data

One of the most important challenges is to collect data after the program has been implemented using a variety of follow-up methods. The typical methods are:

Increasing Response Rates

Improving response rates is a critical issue for post-program collection. When used consistently, the following techniques can achieve 70-80 percent response rate for questionnaires, surveys, or action plans:

  • Provide advance communication about the follow-up data collection.

  • Review the instrument at the end of the formal session.

  • Clearly communicate the reason for the evaluation and how the data will be used.

  • Indicate who will see the results.

  • Keep the instrument simple and as brief as possible.

  • Keep responses anonymous–or at least confidential.

Questionnaire Topics for Application and Impact

  • Use of materials, guides, and technology

  • Actions taken by participants

  • Procedures followed

  • Application of knowledge and skills

  • Frequency of use of knowledge and skills

Data Collection Issues

Sources of Information for Program Evaluation

  • Participants

  • Managers of participants

  • Direct reports of participants

  • Peer groups

Factors to Consider when Determining Timing of Follow up

  • Availability of data

  • Ideal time for application (Level 3)

  • Ideal time for business impact (Level 4)

Factors to Consider when Selecting Data Collection Methods Time required for participants

  • Time required for participants’ managers

  • Costs of method

  • Amount of disruption of normal activities

Step 6

Make it Credible: Isolate the Effects of the Program

One of the most critical steps in the process is to isolate the effects of the program on impact data. This identifies the amount of impact directly connected to the program.

While isolating the effects of the program with other influences is sometimes difficult, it is necessary for credibility of the study. Without this step, there is no proof that the program is connected to a business measure.

Step 7

Make it Credible: Convert Data to Monetary Value

To calculate the ROI, improvement in business measures must be converted to money.

This step develops a monetary benefit for one or more impact measures linked to the program. It usually follows the step to isolate the impact of the program.

To calculate the monetary value:

  1. Identify the unit of improvement, e.g. one first aid treatment

  2. Determine the value of each unit (V), e.g. $300, a standard value

  3. Determine the unit performance change (Δ), e.g. 6 incidents per month (experimental vs. control)

  4. Determine the annual performance level change (ΔP), 6 x 12 = 72

  5. Calculate the annual improvement value (V times ΔP), e.g. $300 x 72 = $21,600

Step 8

Make it Credible: Identify Intangible Measures

Intangible benefits are program benefits that you choose not to convert to money. They are measures that cannot be converted to money credibly with minimal resources.

Identifying Intangibles

  1. During needs assessment, the intangibles are sometimes identified as directly connected to the program, and a decision is made not to convert them to monetary values. They are listed as intangibles, but only if they are connected to the program.

  2. In the planning phase of the ROI study, intangible measures are often suggested as outcomes.

  3. During data collection, participants and other stakeholders may offer additional intangibles, usually unintended, that are connected to the program.

  4. Finally, during data analysis, when measures cannot be converted to monetary values credibly with minimum resources, they are listed as intangibles.

Step 9

Make it Credible: Capture Costs of Program

When impact studies are conducted, the total costs of the program are needed for the ROI calculation. The costs must be fully loaded, i.e., must include all direct and indirect costs.

Typical Cost Categories

  • Initial needs assessment and analysis—possibly prorated over the expected life of the program

  • Program design and development—possibly prorated over the expected life of the program

  • Software or equipment—purchase allocated in some convenient way

  • Project or program materials—cost of all materials provided to each participant or consumed in the program

  • Facilities—use of facilities to execute the program

  • Facilitator/coach/coordinator—includes preparation time as well as delivery time

  • Salaries plus benefits—of the participants for the time they are involved in the program

  • Administrative and overhead costs—allocated in some convenient way

  • Evaluation—the costs of the impact or ROI study

Step 10

Make it Credible: Calculate Return on Investment

Return on Investment (ROI) is a financial metric, representing the ultimate measure of program success. Benefit-Cost Ratio (BCR) is the efficient use of funds. Both are calculated using the program benefits and costs.

Basic Formulas

The benefit-cost ratio is the program benefits divided by cost. In formula form, it is:

The return on investment calculation considers the net benefits divided by program costs. The net benefits are the program benefits minus the costs. In formula form, the ROI becomes:

This is the same basic formula used in evaluating capital investments where the ROI is traditionally reported as earnings divided by investment.

The payback period compares total investment (cost) to monetary benefits to calculate the number of years (or percent of a year) needed to pay back the investment. The calculation is:

Step 11

Tell the Story: Communicate Results to Key Stakeholders

Reporting the results of the study is an important step in the ROI Methodology. Properly identifying the audience and providing appropriate information is essential.

Four Audiences Are Essential

  1. The participants directly involved in the program who provide data to the evaluators.

  2. The immediate managers of the participants who need evidence of the success of the program.

  3. The sponsors of the program who need to understand the program’s value to the organization.

  4. The staff/team members who need to know how the ROI evaluation was developed.

Step 12

Optimize Results: Use Black Box Thinking to Increase Funding

This is the final step in the ROI Methodology and is designed to sustain or increase funding.

Use of Data

A challenge with evaluation is using the data appropriately. The uses of evaluation data include improving design and delivery processes, enhancing budgets, and building support and commitment from a variety of groups.

Optimize Results

With the intense competition for resources, it is important to show key funders and supporters the value of programs. Very credible and unmistakable results make a great case for maintaining or increasing funding. However, it starts with the issue of process improvement, as data are collected and used to make changes to improve the program.

Making Adjustments in Programs

The good news is that the causes of failure (or disappointing results) can be identified, and adjustments can be made at different points in the cycle. These adjustments are all aimed at making the program or project more successful and essentially moving it from mediocre or negative results to delivering very positive results.

Influencing Allocation

Fundamentally, ROI is increased by either increasing the monetary benefits of the program (the numerator of the equation) or by decreasing the cost of the program (the denominator). Sometimes, both are necessary.

Cost Versus Investment

An organization has many activities that represent costs, and the perception of executives and administrators about these costs becomes critical. If executives see the activity as an investment with a positive ROI, then there is a reluctance to minimize or reduce it.

Status of Measurement and Evaluation

All programs are not evaluated to every level. How does your use of the levels compare to the recommended use?

More information on implementing the ROI Methodology can be found in this guide!

Did this answer your question?