What is Program Strategy in Cognota?
Welcome to our Program Strategy feature, developed in collaboration with the ROI Institute. Cognota is the only L&D software provider to partner with the ROI Institute to integrate their evidence-based ROI Methodology®, so you can assess learning program effectiveness and business impact with a tested, proven methodology.
Designed to enhance your learning experience within Cognota, this feature integrates the renowned Learning Measurement framework, empowering teams to measure the impact of their learning investments effectively.
With Program Strategy, administrators and program owners can create, manage, and evaluate learning programs with precision.
How to Use Program Strategy
From setting objectives to measuring results for your Programs, this guide will walk you through each step, ensuring a seamless experience regardless of your technical proficiency. Let's dive in!
Accessing Program Strategy
Log in to Cognota and navigate to the “Programs” section.
Click on any Program.
Selecting Strategy:
Click on the “Strategy” tab located in the top right corner of the program you want to work on.
Next, you'll be presented with the ROI Methodology Levels — Levels 0 to 5.
Level 0 - Input: Allows you to capture details about your learners and the business need.
Level 1 - Reaction: Allows you to set and manage objectives, create evaluation plan and document results for how participants should feel about the program.
Level 2 - Learning: Allows you to set and manage objectives, create evaluation plan and document results for the skills and knowledge participants should gain.
Level 3 - Application: Allows you to set and manage objectives, create evaluation plan and document results for behaviour changes created by the program.
Level 4 - Impact: Allows you to set and manage objectives, calculate any Monetary or Intangible impacts that the Program has had, create an evaluation plan and document results.
Level 5 - ROI: Allows you to compare monetary benefits from the program to program costs.
Input:
This level allows you to
Link Requests to the Program Strategy.
Define the business unit the learning is being requested from
Specify the completion date for the learning
Populate important program inputs and drivers such as the payoff opportunity, business need, job performance need, learning need and preference need.
Capture your learner details such as the audience and number of learners
Setting Objectives
Setting Objectives for Levels 1 to 5 (Reaction, Learning, Application, Impact, ROI):
Each level, from Levels 1 to 4, gives you the ability to set and manage objectives. However, the approach for Levels 4 (Impact) and 5 (ROI) differs.
For Levels 1 to 3:
Choose from Reaction, Learning, Application or Impact objectives.
Click on “Add Objective” and follow the on-screen guidance to set effective objectives.
For Level 4 (Impact):
The Impact level allows you to add quantifiable objectives. For example, reducing employee churn by 10%. Click on "Add Objective" to create one.
In addition, you have the ability to account for additional benefits—monetary or intangible—that were created as a result of the program. To create additional benefits, click on "Add Benefit".
For Level 5 (ROI)
Click on "Add Objective" and define the ROI percentage your program should achieve.
Save the objective to establish your financial success target.
Next, click on "Manage Costs" under Program Costs and enter detailed descriptions and values of all costs associated with the program.
Save the inputs to ensure all cost factors are recorded accurately.
Managing Objectives
Managing Objectives (Levels 1 to 3 - Reaction, Learning, Application):
After setting objectives, manage them by selecting “Manage Objective” when you have results to share.
Specify if the objective was met or not.
Select the measurement scale.
Click on "Save".
Managing Objectives (Level 4 - Impact):
After setting objectives and/or benefits manage them by selecting “Manage Objective” or "Manage Benefit" when you have results to share.
When managing an objective, you'll be presented with a Benefits Calculator to quantify your result. Input your unit of measure (e.g. Absences), value of one unit of measure (e.g. cost of avoiding one absence), and the change in performance per month (e.g. number of absences prevented per month).
The system calculates the annual monetary benefit based on the information provided.
Specify if the objective was met or not.
Click "Save".
When managing benefits on the other hand, you'll have the ability to specify if the benefit is monetary or intangible. If monetary, you'd be presented with the Benefits Calculator. Simply follow the steps above. If it is intangible, simply select that option.
Click on "Save".
Managing Objectives (Level 5 - ROI)
After setting a desired ROI percentage for your program, you can click on "Manage Objective" to update the desired ROI value at any time.
Evaluation Plan
Creating Evaluation Plan (Levels 1 to 4):
Develop an Evaluation Plan for each level (Reaction, Learning, Application, and Impact) to measure objectives.
Selecting Measurement Methods: You can select 1 or more measurement methods for each objective. For every measurement method, You must select a data source.
Reaction: You can select from the following:
Data collection method(s)
survey, questionnaire, interview, focus group
Data source(s)
Participants, Facilitators, Other (input custom data source)
Data timing
Learning: You can select from the following:
Measurement method(s)
Questionnaire, Observations, Interviews, Test
Data source(s)
Participants, Facilitators, Managers, Colleagues/Team, Other (input custom data source)
Application: You can select from the following:
Measurement method(s)
Survey, Questionnaire, Observation, Interviews, Focus Groups, Action Planning/Performance Contracting
Data source(s)
Participants, Facilitators, Managers, Colleagues/Team, Other (input custom data source)
Impact: You can select from the following:
Measurement method(s)
Survey, Questionnaire, Observation, Interviews, Focus Groups, Action Planning/Performance Contracting, Performance Monitoring
Data source(s)
Participants, Facilitators, Managers, Colleagues/Team, Direct Reports, Client/Business Partners, External Customers, Other (input custom data source)
Detailed Evaluation Approach: Provide more comprehensive details about your evaluation approach when you select observation as your data source.
Observation:
You can enter observation method
You can add the behaviours you are hoping to see
Results, ROI Calculation & Evaluation Evidence
Capturing Results:
Record results for each objective once the program is complete to assess whether objectives were met and maintain a historical record of program success. Do this by clicking on "Manage Objective" in any of the levels and indicate whether or not you met the objective as well as the measurement scale. Click "Save" once completed.
ROI Calculation (Level 5 - ROI)
Once program costs are inputted and saved, the sustem automatically calculates the ROI by converting the monetary benefits calculated in Level 4 - Impact and your programs costs into a benefit-cost ratio and ROI expressed as a percent. This provides a clear measure of your program's financial performance.
Uploading Evaluation Evidence:
Upload evidence files, such as questionnaire findings, observation videos, or signed action plans, to each level (Levels 1 to 4).
File Types and Downloads:
Ensure files are in the following formats: jpg, png, mp4, mpeg, mp3, pdf, ppt, pptx, xls, xlsx, doc, csv. You can also download evidence files when needed.
The steps above would allow you to effectively utilize the Program Strategy feature to measure the impact of your learning investments and demonstrate the value of your programs.