Training Effectiveness Iso 9001

Training Effectiveness, design and deliveryan ISO-9000 perspective.

Agenda • Overview: • ISO Model • Requirements (9K, 14K, 18K).

• Kirpatrick Model for Training Evaluation. • ISO-10015: Guidelines for Training.

Background

Quality Management System Model CONTINUAL IMPROVEMENT OF THE UNIFIED MANAGEMENT SYSTEM

C uR s qe u t i r o me m ne e ts r

S a t i s f a c t i o n

Management responsibility Resource management

Input

Measurement, analysis, improvement Product realization

Output

Product

C u s t o m e r

6. Resource Management 6.2 Human Resources 6.2.1 General Personnel whose work affect product quality to be COMPETENT on the basis of appropriate education, training, skills and experience

5

6. Resource Management (Contd.)

6.2 Human Resources

6.2.2 Competence, Awareness and Training The organization to

• Determine necessary competence for personnel performing work affecting product quality, • Provide training or take any other actions to satisfy these needs***, • Evaluate effectiveness of the actions taken.***

Ensure that its personnel are aware of • •

relevance and importance of their activities how they contribute to the achievement of the Quality Objectives

Maintain records of education, experience, training and skills *** Changed in 2008 version

6

6. Resource Management (Contd.)

6.2 Human Resources

(Contd)

6.2.2 Competence, Awareness and Training (Guidelines)

Make employees at each relevant function level aware of • • • • •

importance of conformance with the Quality Policy & QMS requirements significant impacts of their work activities on quality, actual or potential benefits of improved personnel performance roles and responsibilities in achieving conformance to quality policy the potential consequences of departure from specified procedures

7

Changes in ISO-9001-2008 Paragraph 6.2 Words, "affecting product quality" changed to "affecting conformity to product requirements”

Paragraph 6.2.2 Sub-clause (b) -"provide training or take other actions to satisfy these needs" changed to "where applicable training needs to be provided to achieve the necessary competence". Sub-clause (c) – obligation to ensure that the staff attained the expected competence.

ISO-14000 & OHSAS-18001 4.4.2 Training Awareness & competence • Identify training needs • Establish training procedures • Importance of conformance • Consequences of departure from procedures

• Personnel competent by education, training, and/or experience 9

4.4.2 Training Awareness & competence

• Training department completes Training Needs Analysis (TNA) and training schedule, reviews at least annually and revises as necessary • Permanent on-site contractors included in training program • Training records maintained

10

Effectiveness Extent to which planned activities are realized and planned results achieved. ISO 9000:2000 3.2.14

ISO 9001 specifies requirements for a quality management system that can be used for internal application by organizations, or for certification, or for contractual purposes. It focuses on the effectiveness of the quality management system in meeting customer requirements. ISO 9001: 2000 0.3

Training Design: ISO-9001

Part of Product Realization Process, Clause 7.3  Training Design Planning  Design Inputs  Design Output  Design Review  Design Verification  Design Validation  Control of changes 12

7. Product Realization (Contd.) 7.3 Design & Development (D/D) 7.3.1 Design and Development Planning

• Plan and control Design and Development of the product • During planning, determine • Stages of design and development • Review, verification and validation appropriate to each Design and Development stage • Responsibilities and authorities for Design and Development • Manage interface between different involved groups to ensure effective communication and clear responsibilities • Update planning output, as appropriate, as Design and Development progresses

13

7. Product Realization (Contd.) 7.3 Design & Development (D/D)

(Contd)

7.3.2 Design & Development Inputs ❚ Determine the inputs related to product requirements and maintain records ❚ Design & Development inputs to include ❙ Functional and performance requirements ❙ Applicable statutory and regulatory requirements ❙ Where applicable, Information derived from previous similar designs ❙ Other requirements essential for Design and Development

❚ Review inputs for adequacy. Requirements to be complete, unambiguous and non-conflicting

14

7. Product Realization (Contd.) 7.3 Design & Development (D/D)

(Contd)

7.3.3 Design & Development Outputs ❚ Outputs of D/D provided in a form that enables verification against D/D inputs. This shall be approved prior to release ❚ D/D Outputs shall : ❙ meet D/D input requirements ❙ provide appropriate information for purchasing, production and for service provisions ❙ contain or reference product acceptance criteria ❙ specify product characteristics essential for safe and proper use 15

7. Product Realization (Contd.) 7.3 Design & Development (D/D)

(Contd)

7.3.4 Design & Development Review ❚ At suitable stages, conduct systematic D/D reviews ❙ to evaluate the ability of results of D/D to fulfill requirements ❙ to identify any problems and proposed necessary actions ❚ Involve and include representatives of functions concerned with design stage(s) being reviewed ❚ Records of results of reviews and necessary actions to be maintained

7.3.5 Design & Development Verification

❚ Perform verification to ensure that D/D outputs have satisfied D/D input requirements ❚ Records of results of verification & necessary actions to be maintained

16

7. Product Realization (Contd.) 7.3 Design & Development (D/D)

(Contd)

7.3.6 Design & Development Validation ❚ Perform D/D validation in accordance with planned arrangements (see 7.3.1) to ensure ❙ Resulting product is capable of fulfilling the requirements for the specified or known intended use or application

❚ Wherever practicable, validation to be completed prior to delivery or implementation of the product ❚ Records of results of validation and necessary actions to be maintained

17

7. Product Realization (Contd.) 7.3 Design & Development (D/D)

(Contd)

7.3.7 Control of Changes ❚ Changes in D/D to be identified and records maintained ❚ The changes to ❙ be reviewed, verified and validated, as appropriate ❙ be approved before implementation ❙ include evaluation of the effect of changes on constituent parts and delivered products

❚ Records of the results of validation and necessary actions to be maintained 18

Training Evaluation

6 - 19

How the training is evaluated? Factors affecting evaluation?

Evaluation - Myths I Can’t Measure the Result of My Training Efforts. I Don’t Know What Information to Collect. If I Cannot Calculate the ROI, the Evaluation Is Useless. My CEO Does not Require Evaluation, So, Why Should I Do It ? There Are Too Many Variables Affecting the Behavior Change for Me to Evaluate the Impact of Training. Evaluation Will Lead to Criticism. I Don’t Need to Justify My Existence; I Have a Proven Track Record.

Factor

Change potential Importance

Scale Purpose of training

How Factor Influences Type of Evaluation Design

Can program be modified?

Does ineffective training affect customer service, product development, or relationships between employees? How many trainees are involved? Is training conducted for learning, results, or both?

Questions we must answer … How do participants feel about our training program? Are participants learning? Is their learning transferring to the job? Does the organization benefit from our training efforts?

Donald Kirpatrick’s Model for Training Evaluation

Finding answers … Kirkpatrick’s 4 Levels of Evaluation Level 1: Reaction How do participants feel about our training program?

Level 2: Learning Are participants learning?

Level 3: Behavior Is their learning transferring to the job?

Level 4: Results Does the organization benefit? Source: Kirkpatrick,1998

 DONALD

KIRKPATRICK'S 4 LEVELS OF EVALUATING TRAINING

Levels

Description

Comments

Level 1

Reaction

Trainee reaction to the course. Does the trainee like the course? Usually in the form of evaluation forms, sometimes called "smile sheets".

Level 2

Learning

Did trainees learn what Learning can be measured was based on the course by pre- and post tests, objectives? either through written test or through performance tests.

Level 3

Behavior

Trainee behavior changes on the job - are the learners applying what they learned?

Difficult to do. Follow-up questionnaire or observations after training class has occurred. Telephone interviews can also be conducted.

Level 4

Results

Ties training to the company's bottom line.

Generally applies to training that seeks to overcome a business problem caused by lack of knowledge or skill. Examples include reductions in costs, turnover, absenteeism and grievances. May be difficult

Most primitive and widelyused method of evaluation. It is easy, quick, and inexpensive to administer. Negative indicators could mean difficultly learning in the course.

Appeal of Kirkpatrick’s Model: Assesses important areas Widely known Simple framework Easy to explain and understand

However . . . Widely Know ≠ Widely Used Level 1: Often (over 90%) Level 2: Sometimes (less than 35%) Level 3 & 4: Rarely (less than 15%)

Why is this a problem? Level 3 and 4 often perceived as: Difficult to measure Time consuming Beyond the realm of most trainers

Level 1 result does not always mean similar Learning / Transfer / ROI results Source: Pershing & Gilmore, 2004

Other problems … Undermines Management Partnership Training ≠ “Silver Bullet” Training is only one strategy within entire Performance System Level 3 & 4 should include evaluations of entire Performance System - not just training

Lacks Performance System Focus What about rest of Performance Environment? What factors impede / enable usage of training?

Feedback Goes to Wrong People Feedback to training function only is incomplete Must include Performance Environment owners

Source: Brinkerhoff & Dressler, 2002

In Summary: An evaluation tool must be integrated into the training programs. Kirkpatrick’s Four Levels of Evaluation is well known, but has limitations: Unlikely completion of all 4 Levels Lacks performance system focus

ISO-10015 Guidelines for Training

Background of ISO-10015 22 country representatives developed the draft text over several years, culminating in the publication of ISO 10015 issued December 1999. The advantages: • based on the process oriented concepts of the new 9000:2000 ISO family of standards. • Offers specific guidance for training design, review and implementation. • Provides context to organization's needs.

32

Key features of ISO 10015 1. Linking

training

performance

investment

with

company

 

•

Key client is the organization, not only the persons being trained. (ROI Approach)

•

A company has to recognize first what is the performance challenge it faces and the causes for it.

•

ISO 10015 offers a clear road map to connect training to performance goals and use it for individual and group performance improvement.

2. Organizing training on the basis of pedagogical principles and processes •

Training as a tool to cover an identified performance gap.

•

Establishing an appropriate training design and effective learning processes.

•

Ensures that training uses resources (finances, time and energy) optimally.

Training: for organizational needs

4.1Training is 4-step process

For selecting and implementing training to close the gaps between required and existing competence, management should monitor the following stages:

a) defining training needs; b) designing and planning training; c) providing for the training; d) evaluating the outcome of training.

The Training Process

4.2 Defining Training Needs

4.2.1 General 4.2.2 Defining the needs of the organisation 4.2.3 Defining the competence needs 4.2.4 Reviewing Competence 4.2.5 Defining Competence gaps 4.2.6 Identifying solutions close to competence gaps 4.2.7 Defining the specifications for training needs

Training Processes-1

Training Processes-2

Training Processes-3

Training Processes-4

4.3 Designing and planning training

4.3.1 General 4.3.2 Defining the constraints 4.3.3 Training methods and criteria for their selection 4.3.4 Training Plan specifications 4.3.5 Selecting a training provider

4.4 Providing the training 4.4.1 General 4.4.2 Providing Support 1. pre-training support 2. training support 3. end-of-training support

4.5Evaluating training outcomes

4.5.1 General 4.5.2 Preparing evaluation report Contents of evaluation report: – specification of training needs – Evaluation criteria and description of sources, methods and schedule for evaluation. – Analysis of data and interpretation – Review of training costs – Conclusions and recommendations for improvement.

5 Monitoring and improving the training process 5.1 General 5.2 validation of training process Monitoring is the key tool: – Monitoring is to be done by competent personnel only. – Monitor records at all stages. Detect nonconformity at all stages. Take preventive and corrective actions. – if procedures followed and requirements met, then update the personal competence records. – if procedures not followed yet requirements are met, revise procedures and update training records. – If procedures followed and requirement not met, either correct the training process or seek a nontraining solution.

• Identify further opportunities for improvement.

Thank You