Module1, Introduction To Development Evaluation

IPDET

Module 1: Introduction to Development Evaluation What? Professional Associations

Origin & History

Principles and Development Evaluation Standards Context OECD DAC Criteria

Introduction • • • • • •

Evaluation, What Is It? Origin and History Evaluation Development Context Built on OECD DAC Criteria Principles and Standards Growth of Professional Associations IPDET

22

OECD Definition of Evaluation Evaluation • the process of determining the worth or significance of an activity, policy, or program • an assessment, as systematic and objective as possible, of a planned, ongoing, or completed intervention IPDET

33

Kinds of Evaluations • Formative – focus on improved performance before and during implementation (project, program or policy)

• Summative – focus on outcomes (consequences)

• Prospective – answer questions • Is this program/project/policy worth evaluating? • Will the gains be worth the effort/resources expended?

IPDET

44

Purpose of Evaluation • • • •

Ethical Managerial Decisional Educative and Motivational

IPDET

55

What to Evaluate? • • • • • • •

Projects Programs Policies Organizations Sectors Thematic Country assistance IPDET

66

Uses of Evaluations • Give feedback on the performance of policies, programs, and projects • Make policies, programs, and projects accountable for how they use public funds • Help stakeholders learn more about about policies, programs, and projects • Carried out around the needs of the primary intended user • Clarify theory of change (logic model) for projects, programs, and policy IPDET

77

Evaluation Provides Information on: • Strategy – – Are the right things being done? • Operations – – Are things being done right? • Learning – – Are there better ways? IPDET

88

Monitoring and Evaluation • Monitoring – routine, ongoing, and internal activity of tracking key indicators – internal activity – used to collect information on a program’s activities, outputs, and outcomes to measure performance of the program

• Evaluation – periodic and time bound – can be internal, external, or participatory – periodic feedback to key stakeholders IPDET

99

Roles and Activities of Professional Evaluators • Internal Evaluation • External Evaluation • Participatory Evaluation

IPDET

10 10

Internal, External, and Participatory Evaluators • Internal evaluators: – usually know more • history, organization, culture, problems, successes

– may be too close

• External evaluators: – have more credibility and specialized skills – independent from administration and financial decisions (continued on next slide)

IPDET

11 11

Internal, External, and Participatory Evaluators (cont.) • Participatory evaluators: – representatives of agencies and stakeholders (including beneficiaries) work together in designing, carrying out and interpreting an evaluation – participatory methods may be used in internal and external evaluations IPDET

12 12

Participatory Evaluation (vs. Internal or External) • Distinction between experts and layperson, researcher and researched, is de-emphasized and redefined • Evaluators act as facilitators and instructors • Others make the assessment • A more radical step away from the model of independent evaluation IPDET

13 13

Evaluator Activities • • • • •

Consult with all key stakeholders Review and/or develop theory of change Design the evaluation Manage evaluation budgets Perform or conduct the evaluation (or contract staff to perform the evaluation) • Identify standards for effectiveness • Collect, analyze, interpret, and report on data and findings IPDET

14 14

1980s to now

1950s-60s 1957 1940 1900 1880 1600 2000 BC

into ion ty i ans Exp al activ s glob tat u ns sio fes Pro tine pe) rou Euro re Mo and ( US ik n u tn c t io k s Sp str u - B a n co n Re W W o l s er af t cho da) al S C a na dic Me a n d d s (U S an on ram ati og uc Pr Ed cial So

d

s rn de i o n M o a l ua t Ev

n ta yp E g i na Ch

15 15

IPDET

mid 1970-80

Timeline

Origins of Development Evaluation • Audit tradition • Social science tradition

IPDET

16 16

Changing Development Concepts Decade Objectives Approaches Discipline 1950s Reconstruction Technical assistance Engineering

1960s 1970s

Growth

Projects

Finance

Basic needs

Sector investment

Planning

1980s

Adjustment

Adjustment lending Neoclassical economics

1990s

Capacity

Country assistance

Multi-disciplinary

2000s

Poverty

Partnerships

Results-based management

reduction

IPDET

17 17

Significant Changes • From partial development to more comprehensive development • Toward global approaches to development • From individual efforts to coordinated, participatory development • Toward using partnerships for largescale development challenges IPDET

18 18

Evolution and Expanding Roles Financial Financial Audit Audit

Evaluator Evaluatoras as Accountant Accountant

Performance Performance Audit Audit

Impact Impact Evaluation Evaluation

Evaluator Evaluatoras as Researcher Researcher

IPDET

Learning Learning Focused Focused

Evaluator Evaluatoras as Facilitator Facilitator

19 19

Development Evaluation: • A sub-discipline of classical evaluation • Uses a variety of methodologies and practices • Mixed methodologies work best

IPDET

20 20

Development Assistance Committee (DAC) Criteria for Evaluating Development Assistance • Relevance – The extent to which the aid activity is suited to the priorities and policies of the target group, recipient, and donor

• Effectiveness – A measure of the extent to which an aid activity attains its objectives

• Efficiency – Measures the outputs – qualitative and quantitative – in relation to the inputs. (continued on next slide)

IPDET

21 21

DAC Criteria for Evaluating Development Assistance (cont.) • Impact – positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended

• Sustainability – measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn IPDET

22 22

DAC Specific Principles for the Evaluation of Development Assistance • • • • • •

Purpose Impartiality and independence Credibility Usefulness Participation of donors and recipients Donor cooperation

(continued on next slide)

IPDET

23 23

DAC Specific Principles for the Evaluation of Development Assistance (cont.) • Evaluation programming • Design and implementation of evaluations • Reporting, dissemination, and feedback • Application of these principles

IPDET

24 24

Why Standards? • Promote accountability • Facilitate comparability • Enhance reliability and quality of services provided

IPDET

25 25

Evaluation Standards • DAC Evaluation Quality Standards (for test phase application) – http://www.oecd.org/dataoecd/30/62/36596604.pdf

• Program Evaluation Standards – www.eval.org/EvaluationDocuments/progeval.html

• United Nations Norms for Evaluation in the UN System – www.uneval.org/docs/ACFFC9F.pdf (continued on next slide)

IPDET

26 26

Evaluation Standards (cont.) • Standards for Evaluation in the UN System – www.uneval.org/docs/ACFFCA1/pdf

• American Evaluation Association (AEA) Guiding Principles – www.eval.org/Publications/GuidingPrinciples.asp

IPDET

27 27

Evaluation and Independence • Independent evaluation (OECD glossary): – an evaluation carried out by entities and persons free of the control of those responsible for the design and implementation of the evaluation – the credibility of an evaluation depends in part on how independently it has been carried out IPDET

28 28

Criteria of Evaluation Independence • • • •

Organizational independence Behavioral independence Avoidance of conflicts of interest Protection from external influence

IPDET

29 29

Growth of Professional Evaluation Associations • Increasing numbers of associations around the world, especially in the last few years • Demonstrate more value is being given to development evaluation as a profession • Organizations create a support system and allow for professionalism IPDET

30 30

International Organizations • International Organisation for Cooperation in Evaluation (IOCE) • International Development Evaluation Association (IDEAS)

IPDET

31 31

To continue on to the Next Module click here To return to the Table of Contents click here