Training Evaluation Basics

A study of Indian Manufacturing Company: Evaluating Effectiveness of Training Programmes

Corporate Profile 

Bharat Heavy Electrical Ltd. (BHEL) is the largest engineering and manufacturing enterprise of its kind in India.



The first plant of BHEL, set up at Bhopal in 1956, Signaled the dawn of the Heavy Electrical Industry in India.



The Company has 14 manufacturing units, 4 power sector regions, 8 service centers and regional offices, besides Project sites spread all over India and abroad.



BHEL Manufactures over 180 Products under 30 Major Product groups and caters to core sectors of the Indian economy.

Historically Training Often Is • Measured from the perspective of the participants • Not required to demonstrate an effective transfer of learning to the work setting & the impact on key measures • The sole responsibility of line-management Costs accounted for in ways that contribute to indifference from management (true costs or of the HR/training department are often unknown) • Managed by staff who themselves believe that the business effects of training cannot be measured credibly

Generally Accepted Best Practices •

The overall training program must have goals aligned with the organizations business goals

•

Individual training modules must have goals and objects aligned with those of the overall program

•

Training goals must meet the stakeholders needs

•

Baseline or control data is needed to create the most valuable measurements

•

The difficulty of measuring increases with the increased value of the measure

HRD Mission To Promote and inculcate a value based culture utilising the fullest potential of Human Resources for achieving the BHEL Mission

Objective of the study • • • •

To gauge the training effectiveness in the organization, To critically review how effective is the training functions, To review how employees perceive the training, To measure about the adequacy of the training for improving skill & competency of employees.

Training Process : 1. Identify training needs 2. Map the Training Approach 3. Produce effective learning Tools

4. Apply Training Techniques 5. Calculate Measurable Results 6. Track ongoing follow-through

Basic Steps in Training  Assessment  Development  Delivery  Evaluation

For best results, evaluation techniques must be planned at the same time the program is being developed.

How Training Needs Are Identified:  Strategic/Organizational Analysis  Task Analysis  Person Analysis

 Trainings helps to bridge the gap b/w expected and actual performance of an employee  All HOD’s / DRO’s identify the training needs of their employees for the year and get it incorporated in the annual training calendar through their respective training coordinator.  On line personal training database is available.  Training needs of workmen are also identified to ensure their effective Redeployment.  Various Programs for Multi skilling are also organized

Identifying & Matching People’s Competencies with Organization’s Need. 

Based on Identified Needs, Employees in different categories are inducted.

 Identified Competency Gaps are met through internal /external training programs, Training at collaborator’s works, participation in the in-house technical meets, seminars and on the job training (OJT) etc.

Category

Induction Training Programs at BHEL

Training Module

Duration

Workman

*Classroom Trg. *Self Dev. Trg. *Trg at ATIs

42 days

Supervisor

*Classroom Trg. *Self Dev Trg. *On the Job Trg.

50 days

Executive

*Class room Trg. *Orientatation Trg. *Site Visit

1 Year

Type Training

Category

Induction Training Modules

W, S, E

Multi-Skilling Audit/Assessment Skills Managerial Leadership Skills (YMP,AMP,SMP) Executive Skill Orientation Supervisory Skill Orientation Competency Development Needs as Per e-Map

W, S E E

E S E

The Faculty For these Programmes is drawn from corporate Apex Mgmt, Industry leaders, IIMS and other Professional bodies.

Why Measure Effectiveness ? • Is the program achieving its purpose of increasing the knowledge and skills of the employees? • What is management's return on investment - Increased production? Decreased workers’ compensation claims? Regulatory compliance? Adherence to company policies? • Are employees comfortable in the facilities and with the training methods? • Are employees satisfied with the overall program?

Payoff of Training Evaluation • • • • • • •

Adjust Program Design Improve Program Delivery Enhance the Transfer of Learning Eliminate Ineffective Programs Expand Successful Programs Improve Satisfaction with Stakeholders Recognize and Reward Participants

Payoff of Training Evaluation • • • • • •

Set priorities for Learning & Development Reinvent Learning & Development Budget Justification Build Support from Managers Develop Norms and Standards Expand Implementation to other areas

Training Evaluation Different stages in Training evaluation: • Input, • Process • Output (post training evaluation) This will help in aligning training needs with organisational needs & meet the emerging challenges.

Pre-Training Evaluation Bar diagram no.i depicting the respondents views on the Pre Training Evaluation 80 70

71.6 66.6

65

60 50.5 46.1 42.2

%

50 40 30

53.3

51.5

50.5

43.3

Moderate

37.7

33.8

36.8

weak

26.6

25 20.5

20 10

15.5 10

11.6

7.7

11.6 5.5

6.6

9.8

0 1

2

3

4 Sl.no.

5

6

7

strongest

8

During Training Bar Diagram No.2 Depicting the respondents View's on During Training 70 62.2 60

53.8 49.4

50

48.3

46.6 45.5

38.3

35.5

47.7 42.2 Strongest

31.6

%

40

37.7

45.5 45

Moderate

30

Weak

20 10

15

13.3 9.4

8.3

6.1

7.7

10

0 1

2

3

4 Sl.no.

5

6

7

Post-Training Evaluation Bar diagram no.3 dipecting the Respondents Views on post Training Evaluation 60

40

50

48.8

50

43.8 37.7

37.7

46.1 42.2 38.8

37.2

35.5

37.2 36.6

%

Moderate 30

26.1 22.2

20

Weak

18.3 15

13.8

12.7

10 0 1

2

3

4 Sl. no.

Strongest

5

6

Evaluation by HRDC… • Level 1: Reaction (IRQ)  To what extent were the course objectives met?  Was the content relevant to the job?  Were the instructional strategies effective?  Did the pre course material prepare participants for the training?  What suggestions trainee have for improving the program?  Should the organization continue to offer it?

Evaluation by HRDC… • Level 2 : Learning  What knowledge, skills or attitudes improved as a result of the learning intervention?  To what extent there was a shift in knowledge?  Did the instructional strategies allow for practice & demonstration?  Did the training content prepare participant for successful learning?

Evaluation by HRDC… • Level 3 : Behavior This evaluation includes the manner and extent to which the trainee has applied his learning's to his Job. What part of the course content are being used on the job? How is participant’s manager supporting the use of the new knowledge, skills and abilities on the job?

Evaluation by HRDC… • Level 4: Results; Organizational Impact

This evaluation measures the use of training, learning & change in the Job behavior of the department/ organization in the form of  increased productivity,  Quality,  Morale,  Sales Turnover etc.

Evaluating Trainer’s Effectiveness. •Feedback from Trainees through Feedback Forms, •Informal discussions during Training, •Through Questionnaire's & meetings. •Performance appraisal during on the Job.

References •

• • • • •

Boudreau J.W, Ramstad. P.M. (2008). “From ‘Professional Business partner’ to ‘Strategic Talent header’; ‘What’s next for human resource Management.” Working paper O2-10, CAHRS/ Cornell University. Ithaces.NY. Cahran.R(2006), “Why it’s a small world after all.” Business week, No.3995, R.Lynton U.Pareek,”Training & Development,” New Delhi, Sage Publications, 2002 Goldstein I.L, Ford J.K, (2002) “Needs Assessment & the learning Environment.” Training organizations 4th edition, Page 35-37, Denise M.Ruggieri, (2003): Training Needs Assessment, “What , Why & How to Assess Training Needs,” For about.com Nickols, Frederick W(1982,April)“Training : a Strategic view” NSPI journal

References…. • • • • • • • •

Phillips J.K.(2001)” Evaluation Models and Design Issues,” Handbook of Training Evaluation and Measurement method, page 51-53. Jackson, S. & Kulp ,M.J(1979).” Designing guidelines for Evaluating the outcomes of management Training,” ASTD,Washington D.C.,p.p7-8. Kirkpatrick, D.L,”Evaluation of training in training & development Handbook, edited by R.L.craig,Mcgraw-Hill,1976. Kirkpatrick, D.L., Evaluating Training Programs : The four levels,Berrett- Koehler 1996. Bartam,S.and Gibson,B.,Training Need Analysis, 2nd edition,Gower,1997. Anett,Duncan,stammers and Gray,Task Analysis,Training information Paper 6 , HMSO,1971. Bartram,S. and Gibson ,B.,evaluating Traning,Gower, 1999. Bee, Frances and Ronald, Training Needs analysis and Evaluation, Institute of Personnel and Development,1994.

Thank you