Measuring Roi Of Training
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Measuring Roi Of Training as PDF for free.
More details
- Words: 3,124
- Pages: 67
Measuring ROI of Training
www.exploreHR.org
1
Agenda 1. Measuring the Effectiveness of Training Program 2. Measuring ‘Return on Investment’ of Training 3. Enhancing the Effectiveness and ROI of Training If you find this presentation useful, please consider telling others about our site (www.exploreHR.org)
www.exploreHR.org
2
Measuring the Effectiveness of Training Program
www.exploreHR.org
3
Training Process Training Need Analysis
What are the training needs for this person and/or job?
www.exploreHR.org
Training Objectives
Objective should be measurable and observable
Training Delivery
Techniques include onthe-jobtraining, action learning, etc.
Training Evaluation
Measure reaction, learning, behavior, and results
4
The Four Levels of Evaluation Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application
Four Levels of Training Effectiveness
Level 4 – Business Impact www.exploreHR.org
5
The Four Levels of Evaluation Evaluate trainees’ reactions to
Level 1 -
the program. Did they like the
Reaction
program? Did they think it worthwhile? Test the trainees to determine if
Level 2 -
they learned the principles,
Learning
skills, and facts they were to learn.
www.exploreHR.org
6
The Four Levels of Evaluation Ask whether the trainees’ behavior on the job changed because of the training program. For example, are employees in the store’s complaint department more courteous toward disgruntled customers than previously?
What final results were achieved in terms of the training objectives previously set? Did the number of customer complaints about employee drop? Did the reject rate improve? Was turnover reduced, and so forth. www.exploreHR.org
Level 3 – Behavior Application
Level 4 – Business Impact
7
The Four Levels of Evaluation Value of Information Level I.
Reaction
Least valuable
Frequency of Use
Difficulty of Assessment
Frequent
Easy
Infrequent
Difficult
II. Learning III. Behavior IV. Results
www.exploreHR.org
Most valuable
8
Level 1 - Reaction Evaluate trainees’ reactions to the program: Did they like the program? Level 1 Reaction
Did they like the facilitators? Did they like the training accommodation and facilities?
www.exploreHR.org
9
Guidelines for Evaluating Reaction 1. Determine what you want to find out 2. Design a form that will quantify reactions 3. Encourage written comments and suggestions 4. Get 100 percent immediate response 5. Get honest response 6. Develop acceptable standards 7. Measure reactions against standards, and take appropriate action 8. Communicate reaction as appropriate www.exploreHR.org
10
Please give us your frank reactions and comments. They will help us to evaluate
Sample of Reaction Form
this program and improve future programs. Program : Facilitator : 1. How do you rate the subject (interest, benefit, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?) a. Excellent b. Very Good c. Good d. Fair e. Poor 3. How do you rate the facilities? (comfort, convinience, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 4. What would have improved the program? ………………………………………………………………………………… ………………………………………………………………………………… …………………………………………………………………………………
www.exploreHR.org
11
Level 2 - Learning Measuring learning means determining one or more of Level 2 Learning
the following : • What knowledge was learned? • What skills were developed or improved? • What attitudes were changed?
www.exploreHR.org
12
Guidelines for Evaluating Learning 1. Use a control group if practical 2. Evaluate knowledge, skills and/or attitudes both before and after the program 3. Use a paper-and-pencil test to measure knowledge 4. Use a performance test to measure skills 5. Get 100 percent immediate response 6. Use the results of evaluation to take appropriate action
www.exploreHR.org
13
Guidelines for Evaluating Learning Pretest and Posttest Scores on Change Management Training Example : Experimental Group
Control Group
Pre Test Score
45
46
Post Test Score
55
48
Gain
10
2
www.exploreHR.org
14
Level 3 – Behavior Application
Level 3 – Behavior Application
www.exploreHR.org
•
The frequency of application of new skills/knowledge/ attitudes (on the job)
•
The effectiveness of the skills/knowledge/ attitudes (as applied on the job)
15
Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for behavior change and application to take place 3. Evaluate both before and after the program if practical 4. Survey and/or interview one or more of the following : trainees, their immediate supervisor, their subordinates, and others who often observe their behavior 5. Get 100 percent response or a sampling 6. Repeat the evaluation at appropriate times 7. Consider cost versus benefits www.exploreHR.org
16
Example of Survey to Measure Behavior Application Instruction: The objective of this questionnaire is to determine the extent to which those who attended the recent program on Leadership have applied the principles and techniques that they learned there to the job. Circle the answer that you consider appropriate for each question. 5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less
Time and energy spent after the prgram compared to time and energy spent before the program Understanding and Motivating 1. Trying to understand my subordinates
5
4
3
2
1
2. Listening to my subordinates
5
4
3
2
1
3. Praising good work
5
4
3
2
1
5
4
3
2
1
5. Asking my subordinates for their ideas
5
4
3
2
1
6. Applying "Management by Walking Around"
5
4
3
2
1
4. Talking with subordinates about their family and personal interests
www.exploreHR.org
17
Level 4 – Business Results •
Level 4 – Business Results
www.exploreHR.org
Indicate the extent to which you think this program has influenced each of these measures in your work unit, department, or business unit: • Productivity • Quality • Customer Response Time • Cost Control • Employee Satisfaction • Customer Satisfaction • Other 18
Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for results to be achieved 3. Measure both before and after the program if practical 4. Repeat the evaluation at appropriate times 5. Consider cost versus benefits 6. Be satisfied with evidence if proof is not possible
www.exploreHR.org
19
Performance Indicators HARD DATA INDICATORS •
Downtime duration
•
Number of defect products
•
Sales volume
•
Production unit
•
Customer satisfaction index
•
Response time to orders
•
Number of accidents at work
•
Others
www.exploreHR.org
20
Performance Indicators SOFT DATA INDICATORS (intangible impacts) •
Job satisfaction
•
Conducive working relationship
•
Effective communication
•
Stress rate
•
Quality in decision-making
www.exploreHR.org
21
Example : Measuring Training Results 200.00
Program : TQM Training Results after
3
150.00
120 units
months of training, number of defects
80 units
100.00
dropped to 80 units/day
50.00
0.00
Before training www.exploreHR.org
After training 22
Example : Measuring Training Results Program :
50.00
Sales Training Results after 3
40.00
30 units
months training, training number of sales
30.00
per salesman increase to 30
20 units 20.00
units/month. 10.00
0.00
Before training www.exploreHR.org
After training 23
Measuring Return on Investment of Training
www.exploreHR.org
24
Level 5 : Return on Investment of Training Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Level 5 – Return on Investment of Training
www.exploreHR.org
25
Criteria for Selecting Programs for Levels 4 and 5 Evaluation •
Importance of the program in meeting the organization’s goals
•
Cost of the program
•
Visibility of the program
•
Size of the target audience
•
Extent of management interest
www.exploreHR.org
26
Benefits of ROI of Training •
Measure contribution
•
Set priorities
•
Focus on results
•
Alter management perceptions of training
www.exploreHR.org
27
ROI of Training Model Collect Data
Isolate the Effects of Training
Convert Data to Monetary Values
Identify Intangible Benefits
Calculate ROI of Training
Tabulate Program Costs www.exploreHR.org
28
Return on Investment Formula Net Program Benefits ROI =
Program Costs
X 100
Example : •
Costs per program (25 participants)
$ 88,500
•
Benefits per program (1st year)
$230,625
$ 230,625 – 88,500 ROI =
X 100 $ 88,500
ROI = www.exploreHR.org
161 % 29
Collecting Data
Collect Data
www.exploreHR.org
•
Identify appropriate performance indicators
•
Develop a collection plan
30
Example of Performance Indicators
Output
• • • • • • •
Units produced Items sold Work backlog New accounts opened Productivity Inventory turnover Etc.
www.exploreHR.org
Time
• • • • • • •
Equipment downtime Overtime Time to project completion Processing time Repair time Lost time days Etc 31
Example of Hard Indicators Cost
• • • • • •
Unit costs Variable costs Overhead costs Operating costs Number of cost reduction Etc.
www.exploreHR.org
Quality
• • • • • • •
Scrap Waste Rejects Error rates Rework Product defects Product failure
32
Example of Performance Results •
Some performance results after training program: •
Scrap was reduced from 11 % to 7.4 %
•
Absenteeism was reduce from 7 % to 3.25 %
•
The annual turnover rate was reduced from 30 % to 16 %
•
Lost time accidents were reduced 95 %
www.exploreHR.org
33
Isolating the Effects of Training Using Control Group Methods to Isolate the Effects of Training
Trend Lines Participants Estimate Supervisors of Participants Estimate
www.exploreHR.org
34
Isolating the Effects of Training Using Control Group
www.exploreHR.org
•
A control group arrangement can be used to isolate training impact.
•
With this strategy, one group receives training, while another, similar group does not receive training.
•
The difference in the performance of the two groups is attributed to the training program.
35
Isolating the Effects of Training Trend Lines
www.exploreHR.org
•
Trend lines are used to project the values of specific output variables if training had not been undertaken.
•
The projection is compared to the actual data after training, and the difference represents the estimate impact of training.
36
Trend Lines Analysis Actual sales performance
Volume of Sales
At the beginning of May, a Sales training Program session was held
The difference represents the estimate impact of training.
Trend Projection
Jan
Feb
www.exploreHR.org
Mar
Apr
May
Jun
Jul
Aug
37
Isolating the Effects of Training Participants and Supervisors of Participants Estimate of Training’s Impact
www.exploreHR.org
•
This method rests on the assumption that participants (and their supervisors) are capable of estimating how much a performance improvement is related to the training program.
•
Because their actions have produced the improvement, participants (and their supervisors) may have very accurate input on the issue.
•
They should know how much of the change was caused by applying what they have learned in the program. 38
Isolating the Effects of Training Participants and Supervisors of Participants Estimate of Training’s Impact
www.exploreHR.org
•
Typical Questions to Estimate : •
What percent this improvement can be attributed to the application of skills/techniques/knowledge gained in the training program?
•
What confidence do you have in this estimate, expresses as a percent?
•
What other factors contributed to this improvement in performance? 39
Isolating the Effects of Training Example of a Participant’s Estimation Factors Which Influenced Improvement
No.
Percent Improvement Caused by
Confidence Expressed as a Percent
1
Training Program
50%
70%
2
Change in Procedures
10%
80%
3
Adjustment in Standards
10%
50%
4
Revision to Incentive Plan
20%
90%
5
Increased Management Attention
10%
50%
6
Other
-
-
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % www.exploreHR.org
40
Isolating the Effects of Training Example of a Participant’s Estimation Factors Which Influenced Improvement
No.
Percent Improvement Caused by
Confidence Expressed as a Percent
1
Training Program
50%
70%
2
Change in Procedures
10%
80%
3
Adjustment in Standards
10%
50%
4
Revision to Incentive Plan
20%
90%
5
Increased Management Attention
10%
50%
6
Other
-
-
•
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % • This adjusted percentage is then multiplied by the actual amount of improvement (post-program minus pre-program value) to isolate the portion attributed to training • The adjusted improvement is now ready for conversion to monetary values, and used in the return on investment www.exploreHR.org
41
Converting Data to Monetary Values Converting Data to Monetary Values
•
Steps to Convert Data to Monetary Values
2. Focus on a unit of improvement 3. Determine a value of each unit 4. Calculate the change in performance data 5. Determine an annual amount of change 6. Calculate the annual value of improvement
www.exploreHR.org
42
Steps to Convert Data to Monetary Values An example to illustrate the steps to convert data to monetary values
Steps
Illustration
1. Focus on unit improvement
•
One grievance reaching step two in the four-step grievance resolution process
2. Determine a value of each unit
•
Using internal experts, the cost of an average grievance was estimated to be $ 6,500 when considering time and direct costs (V = $ 6,500)
www.exploreHR.org
43
Steps to Convert Data to Monetary Values Steps
Illustration
3. Calculate the change in performance data
•
4. Determine an annual amount for the change
•
www.exploreHR.org
•
Six months after the program was completed, total grievances per month reaching step two declined by ten. Seven of the then grievance reductions were related to the program as determined by supervisors (isolating the effects of training)
Using the six month value, seven per month (grievance reductions), yields an annual improvement of 84 (7 x 12 months) 44
Steps to Convert Data to Monetary Values Steps 5. Calculate the annual value of improvement
www.exploreHR.org
Illustration •
Annual value = 84 x $ 6,500 = $ 546,000
45
Tabulating Cost of the Program Tabulating cost of the program
www.exploreHR.org
•
Tabulating the costs involves monitoring or developing all of the related costs of the program targeted for the ROI calculation.
46
Tabulating Cost of the Program Tabulating cost of the program
www.exploreHR.org
•
Cost components that should be included are : •
The cost to design and develop the program
•
The cost of all program materials provided to each participant
•
The cost for facilitator
•
The cost of the facilities of the training program
47
Tabulating Cost of the Program Tabulating cost of the program
www.exploreHR.org
•
Cost components that should be included are : •
Travel, lodging, and meal costs for the participants
•
Salaries, plus employee benefits of the participants who attend the training
•
Administrative and overhead costs of the training function, allocated in some convenient way 48
An Example to Illustrate ROI analysis Average Weekly Sales Post Training Data Weeks after training
Trained Group
Control Group
1
US$ 9,723
9,698
2
9,978
9,720
3
10,424
9,812
13
13,690
11,572
14
11,491
9,683
15
11,044
10,092
12,075
10,449
Average for weeks 13, 14, 15
www.exploreHR.org
49
An Example to Illustrate ROI analysis Annualized Program Benefits Data
US$
Average weekly sales Trained Group
12,075
Control Group
10,449
Increase
1,626
Profit Contribution from Training 2 %
32.50
(Training Impacts) Total weekly improvement
1,495
(32.5 x 46 participants) Total annual benefit
71,760
($ 1,495 x 48 weeks) Note : 46 participants were still in job after 3 months www.exploreHR.org
50
An Example to Illustrate ROI analysis Cost Summary : An Illustration Cost
US$
Facilitation fees
11,250
Program materials : $ 35 x 46
1,610
Meals : 3 days x $ 28 x 46
1,288
Participant salaries plus benefit (35 %) Coordination and Evaluation
12,442 2,500
Total Cost
www.exploreHR.org
ROI (%): $ 71,760 - $ 29,090
=
x 100 $ 29,090
= 146 %
29,090
51
Enhancing Training Effectiveness
www.exploreHR.org
52
How Effective is Your Training Program? Broad and Newstrom (1992) report studies have shown less than 30% of what is actually taught transfers to the job in a way that enhances performance. performance Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: AddisonWesley.
www.exploreHR.org
53
Source of Barriers to Training Transfer •
Lack of reinforcement on the job
•
Interference from immediate (work) environment
•
Nonsupportive organizational culture
•
Trainees’ perception of impractical training programs
•
Separation from the inspiration or support of the trainer
www.exploreHR.org
54
The Transfer Partnership Trainee recognizes need for new skills
Trainee
Trainee
Trainer
Trainer designs and/or delivers learning experiences
Trainee
Trainer
Manager supports learning and application on the job
Manager www.exploreHR.org
55
The Transfer Matrix Time Periods Before
During
After
Role Players
Manager
Trainer
Trainee
www.exploreHR.org
56
Before Training •
Build transfer of training into supervisory performance standards
•
Collect baseline performance data
•
Involve supervisors and trainees in needs analysis process
•
Involve trainees in program planning
•
Brief trainees on the importance of the training (course objective, content, process, and application on the job)
•
Review instructional content and materials
•
Plan to participate in training sessions
•
Encourage trainees attendance at all sessions
Manager
www.exploreHR.org
57
Before Training •
Align the training plan with the organization's strategic plan
•
Systematically design instruction
•
Provide proactive opportunities
•
Design a peer coaching component for the program and its follow-up activities
Trainer
www.exploreHR.org
58
Before Training Trainee
www.exploreHR.org
•
Provide input into program planning
•
Actively explore training options
•
Participate in advance activities
59
During Training Manager
www.exploreHR.org
•
Prevent interruptions
•
Transfer work assignment to others
•
Monitor attendance and attention to training
•
Recognize trainee participation
•
Participate in transfer action planning
•
Review information on employee in training
•
Plan assessment of transfer of new skills to the job
60
During Training Trainer
www.exploreHR.org
•
Develop application-oriented objectives
•
Answer the “WIIFM” question
•
Manage the unlearning process
•
Provide realistic work-related tasks
•
Give individualized feedback
•
Provide job performance aid
61
During Training Trainee
www.exploreHR.org
•
Maintain an ideas and application notebook
•
Participate actively
•
Form support groups
•
Plan for applications
•
Create behavioral contracts
62
After Training Manager
www.exploreHR.org
•
Plan trainees’ reentry
•
Provide opportunities to practice new skills
•
Have trainees participate in transfer-related decisions
•
Reduce job pressures initially
•
Give positive reinforcement
•
Schedule trainee briefings for co-workers
•
Set mutual expectations for improvement
•
Arrange proactive (refresher) sessions
•
Provide and support the use of job aids
63
After Training Trainer
www.exploreHR.org
•
Provide follow-up support
•
Conduct evaluation surveys and provide feedback
•
Develop and administer recognition system
•
Provide refresher/problem-solving sessions
64
After Training Trainee
www.exploreHR.org
•
Practice self-management
•
Review training content and learned skills
•
Develop a mentoring relationship
•
Maintain contact with training ‘buddies’
65
Recommended Further Readings •
Donald Kirkpatrick, Evaluating Training Programs : The Four Levels, Berrett-Koehler Publishers
•
Jack J. Phillips and Patricia Phillips, In Action : Measuring Return On Investment, American Society for Training & Development
•
Mary Broad and John Newstrom, Transfer of training: Action Packed Strategies to Ensure High Payoff from Training Investments, AddisonWesley.
www.exploreHR.org
66
End of Material
www.exploreHR.org
67
www.exploreHR.org
1
Agenda 1. Measuring the Effectiveness of Training Program 2. Measuring ‘Return on Investment’ of Training 3. Enhancing the Effectiveness and ROI of Training If you find this presentation useful, please consider telling others about our site (www.exploreHR.org)
www.exploreHR.org
2
Measuring the Effectiveness of Training Program
www.exploreHR.org
3
Training Process Training Need Analysis
What are the training needs for this person and/or job?
www.exploreHR.org
Training Objectives
Objective should be measurable and observable
Training Delivery
Techniques include onthe-jobtraining, action learning, etc.
Training Evaluation
Measure reaction, learning, behavior, and results
4
The Four Levels of Evaluation Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application
Four Levels of Training Effectiveness
Level 4 – Business Impact www.exploreHR.org
5
The Four Levels of Evaluation Evaluate trainees’ reactions to
Level 1 -
the program. Did they like the
Reaction
program? Did they think it worthwhile? Test the trainees to determine if
Level 2 -
they learned the principles,
Learning
skills, and facts they were to learn.
www.exploreHR.org
6
The Four Levels of Evaluation Ask whether the trainees’ behavior on the job changed because of the training program. For example, are employees in the store’s complaint department more courteous toward disgruntled customers than previously?
What final results were achieved in terms of the training objectives previously set? Did the number of customer complaints about employee drop? Did the reject rate improve? Was turnover reduced, and so forth. www.exploreHR.org
Level 3 – Behavior Application
Level 4 – Business Impact
7
The Four Levels of Evaluation Value of Information Level I.
Reaction
Least valuable
Frequency of Use
Difficulty of Assessment
Frequent
Easy
Infrequent
Difficult
II. Learning III. Behavior IV. Results
www.exploreHR.org
Most valuable
8
Level 1 - Reaction Evaluate trainees’ reactions to the program: Did they like the program? Level 1 Reaction
Did they like the facilitators? Did they like the training accommodation and facilities?
www.exploreHR.org
9
Guidelines for Evaluating Reaction 1. Determine what you want to find out 2. Design a form that will quantify reactions 3. Encourage written comments and suggestions 4. Get 100 percent immediate response 5. Get honest response 6. Develop acceptable standards 7. Measure reactions against standards, and take appropriate action 8. Communicate reaction as appropriate www.exploreHR.org
10
Please give us your frank reactions and comments. They will help us to evaluate
Sample of Reaction Form
this program and improve future programs. Program : Facilitator : 1. How do you rate the subject (interest, benefit, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?) a. Excellent b. Very Good c. Good d. Fair e. Poor 3. How do you rate the facilities? (comfort, convinience, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 4. What would have improved the program? ………………………………………………………………………………… ………………………………………………………………………………… …………………………………………………………………………………
www.exploreHR.org
11
Level 2 - Learning Measuring learning means determining one or more of Level 2 Learning
the following : • What knowledge was learned? • What skills were developed or improved? • What attitudes were changed?
www.exploreHR.org
12
Guidelines for Evaluating Learning 1. Use a control group if practical 2. Evaluate knowledge, skills and/or attitudes both before and after the program 3. Use a paper-and-pencil test to measure knowledge 4. Use a performance test to measure skills 5. Get 100 percent immediate response 6. Use the results of evaluation to take appropriate action
www.exploreHR.org
13
Guidelines for Evaluating Learning Pretest and Posttest Scores on Change Management Training Example : Experimental Group
Control Group
Pre Test Score
45
46
Post Test Score
55
48
Gain
10
2
www.exploreHR.org
14
Level 3 – Behavior Application
Level 3 – Behavior Application
www.exploreHR.org
•
The frequency of application of new skills/knowledge/ attitudes (on the job)
•
The effectiveness of the skills/knowledge/ attitudes (as applied on the job)
15
Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for behavior change and application to take place 3. Evaluate both before and after the program if practical 4. Survey and/or interview one or more of the following : trainees, their immediate supervisor, their subordinates, and others who often observe their behavior 5. Get 100 percent response or a sampling 6. Repeat the evaluation at appropriate times 7. Consider cost versus benefits www.exploreHR.org
16
Example of Survey to Measure Behavior Application Instruction: The objective of this questionnaire is to determine the extent to which those who attended the recent program on Leadership have applied the principles and techniques that they learned there to the job. Circle the answer that you consider appropriate for each question. 5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less
Time and energy spent after the prgram compared to time and energy spent before the program Understanding and Motivating 1. Trying to understand my subordinates
5
4
3
2
1
2. Listening to my subordinates
5
4
3
2
1
3. Praising good work
5
4
3
2
1
5
4
3
2
1
5. Asking my subordinates for their ideas
5
4
3
2
1
6. Applying "Management by Walking Around"
5
4
3
2
1
4. Talking with subordinates about their family and personal interests
www.exploreHR.org
17
Level 4 – Business Results •
Level 4 – Business Results
www.exploreHR.org
Indicate the extent to which you think this program has influenced each of these measures in your work unit, department, or business unit: • Productivity • Quality • Customer Response Time • Cost Control • Employee Satisfaction • Customer Satisfaction • Other 18
Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for results to be achieved 3. Measure both before and after the program if practical 4. Repeat the evaluation at appropriate times 5. Consider cost versus benefits 6. Be satisfied with evidence if proof is not possible
www.exploreHR.org
19
Performance Indicators HARD DATA INDICATORS •
Downtime duration
•
Number of defect products
•
Sales volume
•
Production unit
•
Customer satisfaction index
•
Response time to orders
•
Number of accidents at work
•
Others
www.exploreHR.org
20
Performance Indicators SOFT DATA INDICATORS (intangible impacts) •
Job satisfaction
•
Conducive working relationship
•
Effective communication
•
Stress rate
•
Quality in decision-making
www.exploreHR.org
21
Example : Measuring Training Results 200.00
Program : TQM Training Results after
3
150.00
120 units
months of training, number of defects
80 units
100.00
dropped to 80 units/day
50.00
0.00
Before training www.exploreHR.org
After training 22
Example : Measuring Training Results Program :
50.00
Sales Training Results after 3
40.00
30 units
months training, training number of sales
30.00
per salesman increase to 30
20 units 20.00
units/month. 10.00
0.00
Before training www.exploreHR.org
After training 23
Measuring Return on Investment of Training
www.exploreHR.org
24
Level 5 : Return on Investment of Training Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Level 5 – Return on Investment of Training
www.exploreHR.org
25
Criteria for Selecting Programs for Levels 4 and 5 Evaluation •
Importance of the program in meeting the organization’s goals
•
Cost of the program
•
Visibility of the program
•
Size of the target audience
•
Extent of management interest
www.exploreHR.org
26
Benefits of ROI of Training •
Measure contribution
•
Set priorities
•
Focus on results
•
Alter management perceptions of training
www.exploreHR.org
27
ROI of Training Model Collect Data
Isolate the Effects of Training
Convert Data to Monetary Values
Identify Intangible Benefits
Calculate ROI of Training
Tabulate Program Costs www.exploreHR.org
28
Return on Investment Formula Net Program Benefits ROI =
Program Costs
X 100
Example : •
Costs per program (25 participants)
$ 88,500
•
Benefits per program (1st year)
$230,625
$ 230,625 – 88,500 ROI =
X 100 $ 88,500
ROI = www.exploreHR.org
161 % 29
Collecting Data
Collect Data
www.exploreHR.org
•
Identify appropriate performance indicators
•
Develop a collection plan
30
Example of Performance Indicators
Output
• • • • • • •
Units produced Items sold Work backlog New accounts opened Productivity Inventory turnover Etc.
www.exploreHR.org
Time
• • • • • • •
Equipment downtime Overtime Time to project completion Processing time Repair time Lost time days Etc 31
Example of Hard Indicators Cost
• • • • • •
Unit costs Variable costs Overhead costs Operating costs Number of cost reduction Etc.
www.exploreHR.org
Quality
• • • • • • •
Scrap Waste Rejects Error rates Rework Product defects Product failure
32
Example of Performance Results •
Some performance results after training program: •
Scrap was reduced from 11 % to 7.4 %
•
Absenteeism was reduce from 7 % to 3.25 %
•
The annual turnover rate was reduced from 30 % to 16 %
•
Lost time accidents were reduced 95 %
www.exploreHR.org
33
Isolating the Effects of Training Using Control Group Methods to Isolate the Effects of Training
Trend Lines Participants Estimate Supervisors of Participants Estimate
www.exploreHR.org
34
Isolating the Effects of Training Using Control Group
www.exploreHR.org
•
A control group arrangement can be used to isolate training impact.
•
With this strategy, one group receives training, while another, similar group does not receive training.
•
The difference in the performance of the two groups is attributed to the training program.
35
Isolating the Effects of Training Trend Lines
www.exploreHR.org
•
Trend lines are used to project the values of specific output variables if training had not been undertaken.
•
The projection is compared to the actual data after training, and the difference represents the estimate impact of training.
36
Trend Lines Analysis Actual sales performance
Volume of Sales
At the beginning of May, a Sales training Program session was held
The difference represents the estimate impact of training.
Trend Projection
Jan
Feb
www.exploreHR.org
Mar
Apr
May
Jun
Jul
Aug
37
Isolating the Effects of Training Participants and Supervisors of Participants Estimate of Training’s Impact
www.exploreHR.org
•
This method rests on the assumption that participants (and their supervisors) are capable of estimating how much a performance improvement is related to the training program.
•
Because their actions have produced the improvement, participants (and their supervisors) may have very accurate input on the issue.
•
They should know how much of the change was caused by applying what they have learned in the program. 38
Isolating the Effects of Training Participants and Supervisors of Participants Estimate of Training’s Impact
www.exploreHR.org
•
Typical Questions to Estimate : •
What percent this improvement can be attributed to the application of skills/techniques/knowledge gained in the training program?
•
What confidence do you have in this estimate, expresses as a percent?
•
What other factors contributed to this improvement in performance? 39
Isolating the Effects of Training Example of a Participant’s Estimation Factors Which Influenced Improvement
No.
Percent Improvement Caused by
Confidence Expressed as a Percent
1
Training Program
50%
70%
2
Change in Procedures
10%
80%
3
Adjustment in Standards
10%
50%
4
Revision to Incentive Plan
20%
90%
5
Increased Management Attention
10%
50%
6
Other
-
-
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % www.exploreHR.org
40
Isolating the Effects of Training Example of a Participant’s Estimation Factors Which Influenced Improvement
No.
Percent Improvement Caused by
Confidence Expressed as a Percent
1
Training Program
50%
70%
2
Change in Procedures
10%
80%
3
Adjustment in Standards
10%
50%
4
Revision to Incentive Plan
20%
90%
5
Increased Management Attention
10%
50%
6
Other
-
-
•
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % • This adjusted percentage is then multiplied by the actual amount of improvement (post-program minus pre-program value) to isolate the portion attributed to training • The adjusted improvement is now ready for conversion to monetary values, and used in the return on investment www.exploreHR.org
41
Converting Data to Monetary Values Converting Data to Monetary Values
•
Steps to Convert Data to Monetary Values
2. Focus on a unit of improvement 3. Determine a value of each unit 4. Calculate the change in performance data 5. Determine an annual amount of change 6. Calculate the annual value of improvement
www.exploreHR.org
42
Steps to Convert Data to Monetary Values An example to illustrate the steps to convert data to monetary values
Steps
Illustration
1. Focus on unit improvement
•
One grievance reaching step two in the four-step grievance resolution process
2. Determine a value of each unit
•
Using internal experts, the cost of an average grievance was estimated to be $ 6,500 when considering time and direct costs (V = $ 6,500)
www.exploreHR.org
43
Steps to Convert Data to Monetary Values Steps
Illustration
3. Calculate the change in performance data
•
4. Determine an annual amount for the change
•
www.exploreHR.org
•
Six months after the program was completed, total grievances per month reaching step two declined by ten. Seven of the then grievance reductions were related to the program as determined by supervisors (isolating the effects of training)
Using the six month value, seven per month (grievance reductions), yields an annual improvement of 84 (7 x 12 months) 44
Steps to Convert Data to Monetary Values Steps 5. Calculate the annual value of improvement
www.exploreHR.org
Illustration •
Annual value = 84 x $ 6,500 = $ 546,000
45
Tabulating Cost of the Program Tabulating cost of the program
www.exploreHR.org
•
Tabulating the costs involves monitoring or developing all of the related costs of the program targeted for the ROI calculation.
46
Tabulating Cost of the Program Tabulating cost of the program
www.exploreHR.org
•
Cost components that should be included are : •
The cost to design and develop the program
•
The cost of all program materials provided to each participant
•
The cost for facilitator
•
The cost of the facilities of the training program
47
Tabulating Cost of the Program Tabulating cost of the program
www.exploreHR.org
•
Cost components that should be included are : •
Travel, lodging, and meal costs for the participants
•
Salaries, plus employee benefits of the participants who attend the training
•
Administrative and overhead costs of the training function, allocated in some convenient way 48
An Example to Illustrate ROI analysis Average Weekly Sales Post Training Data Weeks after training
Trained Group
Control Group
1
US$ 9,723
9,698
2
9,978
9,720
3
10,424
9,812
13
13,690
11,572
14
11,491
9,683
15
11,044
10,092
12,075
10,449
Average for weeks 13, 14, 15
www.exploreHR.org
49
An Example to Illustrate ROI analysis Annualized Program Benefits Data
US$
Average weekly sales Trained Group
12,075
Control Group
10,449
Increase
1,626
Profit Contribution from Training 2 %
32.50
(Training Impacts) Total weekly improvement
1,495
(32.5 x 46 participants) Total annual benefit
71,760
($ 1,495 x 48 weeks) Note : 46 participants were still in job after 3 months www.exploreHR.org
50
An Example to Illustrate ROI analysis Cost Summary : An Illustration Cost
US$
Facilitation fees
11,250
Program materials : $ 35 x 46
1,610
Meals : 3 days x $ 28 x 46
1,288
Participant salaries plus benefit (35 %) Coordination and Evaluation
12,442 2,500
Total Cost
www.exploreHR.org
ROI (%): $ 71,760 - $ 29,090
=
x 100 $ 29,090
= 146 %
29,090
51
Enhancing Training Effectiveness
www.exploreHR.org
52
How Effective is Your Training Program? Broad and Newstrom (1992) report studies have shown less than 30% of what is actually taught transfers to the job in a way that enhances performance. performance Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: AddisonWesley.
www.exploreHR.org
53
Source of Barriers to Training Transfer •
Lack of reinforcement on the job
•
Interference from immediate (work) environment
•
Nonsupportive organizational culture
•
Trainees’ perception of impractical training programs
•
Separation from the inspiration or support of the trainer
www.exploreHR.org
54
The Transfer Partnership Trainee recognizes need for new skills
Trainee
Trainee
Trainer
Trainer designs and/or delivers learning experiences
Trainee
Trainer
Manager supports learning and application on the job
Manager www.exploreHR.org
55
The Transfer Matrix Time Periods Before
During
After
Role Players
Manager
Trainer
Trainee
www.exploreHR.org
56
Before Training •
Build transfer of training into supervisory performance standards
•
Collect baseline performance data
•
Involve supervisors and trainees in needs analysis process
•
Involve trainees in program planning
•
Brief trainees on the importance of the training (course objective, content, process, and application on the job)
•
Review instructional content and materials
•
Plan to participate in training sessions
•
Encourage trainees attendance at all sessions
Manager
www.exploreHR.org
57
Before Training •
Align the training plan with the organization's strategic plan
•
Systematically design instruction
•
Provide proactive opportunities
•
Design a peer coaching component for the program and its follow-up activities
Trainer
www.exploreHR.org
58
Before Training Trainee
www.exploreHR.org
•
Provide input into program planning
•
Actively explore training options
•
Participate in advance activities
59
During Training Manager
www.exploreHR.org
•
Prevent interruptions
•
Transfer work assignment to others
•
Monitor attendance and attention to training
•
Recognize trainee participation
•
Participate in transfer action planning
•
Review information on employee in training
•
Plan assessment of transfer of new skills to the job
60
During Training Trainer
www.exploreHR.org
•
Develop application-oriented objectives
•
Answer the “WIIFM” question
•
Manage the unlearning process
•
Provide realistic work-related tasks
•
Give individualized feedback
•
Provide job performance aid
61
During Training Trainee
www.exploreHR.org
•
Maintain an ideas and application notebook
•
Participate actively
•
Form support groups
•
Plan for applications
•
Create behavioral contracts
62
After Training Manager
www.exploreHR.org
•
Plan trainees’ reentry
•
Provide opportunities to practice new skills
•
Have trainees participate in transfer-related decisions
•
Reduce job pressures initially
•
Give positive reinforcement
•
Schedule trainee briefings for co-workers
•
Set mutual expectations for improvement
•
Arrange proactive (refresher) sessions
•
Provide and support the use of job aids
63
After Training Trainer
www.exploreHR.org
•
Provide follow-up support
•
Conduct evaluation surveys and provide feedback
•
Develop and administer recognition system
•
Provide refresher/problem-solving sessions
64
After Training Trainee
www.exploreHR.org
•
Practice self-management
•
Review training content and learned skills
•
Develop a mentoring relationship
•
Maintain contact with training ‘buddies’
65
Recommended Further Readings •
Donald Kirkpatrick, Evaluating Training Programs : The Four Levels, Berrett-Koehler Publishers
•
Jack J. Phillips and Patricia Phillips, In Action : Measuring Return On Investment, American Society for Training & Development
•
Mary Broad and John Newstrom, Transfer of training: Action Packed Strategies to Ensure High Payoff from Training Investments, AddisonWesley.
www.exploreHR.org
66
End of Material
www.exploreHR.org
67
Related Documents
Measuring Roi Of Training
June 2020 6
Tracking Roi Of Training
October 2019 6
Assessing The Roi Of Training
May 2020 7
Assessing The Roi Of Training
May 2020 4
Roi
November 2019 30
Roi
April 2020 27More Documents from "Maintenance Circle"