Module11, Presenting Results

IPDET

Module 11: Presenting Results Intervention or Policy

Analysis & Interpretation

Communication Basics Writing Reports Using Visuals Oral Presentations Peer Review and Meta-evaluations

Presenting Results

Introduction • Communication Basics • Writing Evaluation Reports for Your Audience • Using Visual Information • Making Oral Presentations • Peer-Review and Meta-Evaluation IPDET

22

Purpose of Communicating • The goal is to communicate, not to impress • Make it easy for your reader to get your point (Do not play “hide and seek”) • Keep your purpose and audience in mind

IPDET

33

Choosing Words • Use words that are: – – – – – – –

simple active positive short and concise conversational familiar direct IPDET

44

Communication Strategy • Point of doing an evaluation is so the results can inform: – policymaking – program changes – program replication

• May need multiple products to communicate • Communication strategy helps identify products you need and who for • Develop communication strategy before the evaluation IPDET

55

Questions for Communication Strategy • • • • •

Who will receive what information? In what format? When? Who will prepare the information? Who will deliver the information?

IPDET

66

Strategies for All Phases • Before the evaluation – to ensure everyone is on board

• During the evaluation – to ensure everyone is informed of progress – no surprises

• After the evaluation – to disseminate results, to make decisions IPDET

77

What Kinds of Communication? • Informal – – – –

• Formal

phone email faxes communications

– briefings – presentations – written reports

IPDET

88

Checklist for Strategy Audience

Product

Responsible?

Donor

• formal report

team leader

6/1

Advisory board

• oral briefing

member A

6/1

Local stakeholders

• executive summary • oral briefing

member B

6/1

Program staff

• copy of formal report • executive summary

member C

6/1

Local govt. officials

• oral briefing

team leader

6/5

Participants

• oral briefing

team leader

6/5

Dev. eval. community

• article for publication

team leader

8/1

IPDET

Due Date

99

Feedback • Set up a process to bring stakeholders and evaluator together to discuss: – – – –

findings insights alternative actions next steps

IPDET

10 10

Large Group Discussions • Consider all the stakeholders connected with your program • Identify challenges in communicating evaluation results to different stakeholders

IPDET

11 11

Keys for Writing a Report • Keep it simple • Avoid acronyms • Provide enough information about your research methods so others can judge its credibility • Place technical information in an appendix • Always provide the limitations of the study with cautions to interpretations • Organize around major themes or research questions (continued on next slide)

IPDET

12 12

Keys for Writing a Report (cont.)

• Place major point up front. Lead each paragraph with your point • Leave time to revise, revise, and revise! • Find a person to be a cold reader. Ideally, this should be a detail oriented person, who is looking to make sure every “i” is dotted and every “t” is crossed • Support conclusions and recommendations with evidence IPDET

13 13

The Executive Summary • Provides a quick overview of the study: – – – – –

issues questions methods findings recommendations

IPDET

14 14

Executive Summary Format • The executive summary should be short, two pages are great, more than four are too much • Set up with headings to the left so it is easy for readers to scan the report

IPDET

15 15

Basic Components of the Executive Summary • • • • •

Brief Overview or Introduction Description of the Study Background Major Findings Conclusions/Recommendations

IPDET

16 16

Body of the Report • • • • •

Introduction Description of the evaluation Findings Conclusions Recommendations

IPDET

17 17

Introduction in Body of Report Should contain: • The “hook” that draws readers into the report • Evaluation questions and purpose of the report • Background of the program • Program’s goals and objectives IPDET

18 18

Description of the Evaluation in Body of Report • • • • • •

Evaluation focus Evaluation design Evaluation questions Methodology and strategy for analysis Limitations of the methodology Who was involved and their timeframes IPDET

19 19

Findings in Body of Report • Present data so that your audience can understand them • Present data selectively – what are your most important points? • Organize the findings around study questions, major themes, or program components • Use charts and tables to illustrate and help highlight your major points IPDET

20 20

Conclusions in Body of Report • The final part of your report • Conclusions: – connect to your research questions or evaluation focus – are based on findings and emphasize what the report means – add no new details IPDET

21 21

Recommendations in Body of Report • May be part of the conclusions • Recommendations answer the question: – What do you want the reader to do?

• Recommendations should: – be based on the conclusions – be clear and specific, identifying who should do what and when IPDET

22 22

Recommendation Tracking System (RTS) • RTS is a technique that allows stakeholders to check the implementation of evaluation recommendations • RTS uses a matrix to track: – date of recommendation – who is responsible for taking action – response/ progress IPDET

23 23

Example of an RTS Recommendation

Date

Who is Response/ Responsible Progress

1. 2. 3. 4. IPDET

24 24

Reasons to Use Graphics • Add interest • Communicate information more clearly and effectively • Can be used to “lighten” the density of continuous text • Provide a focal point — attract your audience to key points IPDET

25 25

Characteristics of Good Graphics • simple • communicate without needing text • easily reproduced • clearly labeled • patterns can be distinguished

• culturally appropriate • correctly placed in the text • consistently numbered and titled • sources provided and credit given

IPDET

26 26

Each Visual Aid Must Have: • Title • Number and name of the figure within the section • Provide a list of tables and figures at the beginning of the document

IPDET

27 27

Kinds of Visuals • • • • • •

Pictures and illustrations Organization charts Gantt charts Graphs and data charts Tables Evaluation concepts IPDET

28 28

Pictures and Illustrations • Picture is worth a thousand words • Research conclusions for using pictures and illustrations: – when illustrations are relevant to the content, then moderate to substantial gains can be expected in learning – when illustrations are NOT relevant to the content, or even worse, conflicting, you can expect NO gain in learning and maybe even confusion IPDET

29 29

Connect Narrative to Illustration • If you use a picture or illustration, be sure to use the narrative of the report to tell the audience what they are supposed to see in the picture • Direct them to the picture and tell them what to look for

IPDET

30 30

Types of Pictures and Illustrations • • • •

Maps Sketches Line Drawings Photographs

IPDET

31 31

Data Charts and Graphs • Graphs — visual representation of a relationship between two or more variables • Charts — graphic displays, such as: line charts, pie charts, and bar charts

IPDET

32 32

Effective Charts • Easy to read – Use upper and lower case (not all capital letters) – Only a few type faces

• • • • • • •

Use the message in the title Avoid busy patterns Use white space Keep the chart simple Keep scales honest (proportional) Use title to convey message Put supporting data in an appendix IPDET

33 33

Parts of a Graph Title Vertical or y-axis

Food Costs for Six Months Coordinate

Axis labels

Food Costs in Dollars

2500

2200

2000 1500

1900 1500

1700

Actual value

1400 1200

1000 500 0 Jan

Feb

Mar

Origin

Apr

May

Jun

Month

AxisSource: Fabricated Data 2006 titles Source and year

Grid lines

IPDET

Horizontal or x-axis

34 34

Graphs should Include • • • • •

Title and number Source Year the data were collected Data in chronological order Data portrayed using comparable definitions IPDET

35 35

Format Style for Graphs • No overlapping categories • Patterns or textures that are clear even when photocopied • Patterns clearly labeled using a legend • Have no extra line and patterns – only what is necessary • Avoid black – it is difficult to reproduce accurately • Lettering that does not go below 10pt font IPDET

36 36

Types of Graphs • Line graph • Bar graph • Pie graph or pie chart

IPDET

37 37

Line Graphs • A way to summarize how two pieces of information are related and how they vary • Show data changes over time • Show continuous interval or ratio data

IPDET

38 38

Example of Line Graph – One Line Temperature in C

Average Temperatures for Six Months

40 30

32 28

25

20

24 18

10 0 Feb

Mar

Source: Fabricated data, 2006

Apr

May

Jun

Month

IPDET

39 39

Example of Line Graph – Multiple Lines Grades (out of 100)

Standard 6 Grades for 2006 Legend

100 80 School 1

60

School 2

40

School 3

20 0 1st Qtr

N=523

2nd Qtr

3rd Qtr

4th Qtr

2005 School Year

Source: Fabricated data, 2006

IPDET

40 40

Bar Graphs • Use bars to represent quantities and allow us to compare numbers • Type of bar graphs – single: information about a single variable – multiple: information for more than one variable

• Often show nominal or categorical data IPDET

41 41

Example – Multiple Bar Graph Responses to Questionnaire

Mean Answer to Question

100 80 60

Directors Assistants

40

Laborers

20 0 1

2

Source: Fabricated data, 2006 Question

3

Number

IPDET

4 42 42

Pie Charts • Used to display the size of parts that make up a whole 13% 13% • Example 1st Qtr 17%

2nd Qtr 3rd Qtr 4th Qtr

57% Source: Fabricated data, 2006

IPDET

43 43

Scatter Diagram • Similar to a line graph except coordinates have no lines between them • Used if you want to see if there is a relationship Score on Test

Score s on Te st and Grade Le v e l

100 90 80 70 60 50 40 30 20 10 0 0

1

2

3

4

5

6

7

8

9 10 11 12

Comple te d Grade

Source: Fabricated data, 2006

IPDET

44 44

Poor Example 90

90

80 70

45.9

60 50

30 20

20.4

45

43.9

38.6

30.6

40

46.9

34.6

31.6

27.4 20.4

10 0 1 Directors

2 Assistants

Laborers Assistants Directors

3 Laborers

IPDET

4 Source: Fabricated data, 2006

45 45

When to Use If you want to:

Then choose:

Show trends over time Show percent of a single variable Compare several items Show parts of a whole

Line Chart Single Bar Chart

Show trend or relationships between non-linear data IPDET

Multiple Bar Chart Pie Chart Scatter Diagram 46 46

Tables • Used to present information in an organized manner • Types: – data table – classification table (matrix)

IPDET

47 47

Data Tables • Used to present numerical information • In the report, describe what to see and look for in the data table • Include the year and source

IPDET

48 48

Data Table Principles • Round-off number to no more than

two significant figures • Give averages of rows and columns (as appropriate) to help audience make comparisons • Put the most important comparisons into columns • Too many lines may make it difficult to read IPDET

49 49

Example of Data Table (with lines) Participant number 1

Height

Weight

Age

District

44

30

7.2

North

2

46

35

7.1

East

3

40

20

7.6

North

4

32

22

7.2

South

5

29

23

7.0

South

6

50

38

7.8

North

7

44

30

7.3

West

8

44

28

7.3

West

9

42

30

7.5

East

10

48

45

7.9

South

IPDET

50 50

Example of Data Table (without lines) Participant number 1

Height

Weight

Age

District

44

30

7.2

North

2

46

35

7.1

East

3

40

20

7.6

North

4

32

22

7.2

South

5

29

23

7.0

South

6

50

38

7.8

North

7

44

30

7.3

West

8

44

28

7.3

West

9

42

30

7.5

East

10

48

45

7.9

South

IPDET

51 51

Classification Tables (Matrix) • A tabular layout that shows a list of things organized according to different factors • Can help illustrate complex information

IPDET

52 52

Example of Classification Table (Matrix) Audience

Product

Who is responsible

Due date

Donor

Formal report

Team leader

6/1/06

Advisory board

Oral briefing

Team member A

6/1/06

Local stakeholders

Executive summary Oral briefing

Team member B

6/1/06

Program staff

Copy of formal report Executive summary

Team member C

6/1/06

Local government officials

Oral briefing

Team leader

6/5/06

Participants

Oral briefing

Team leader

6/5/06

Development Evaluation Community

Article for publication

Team leader

8/1/06

IPDET

53 53

Evaluation Concepts • Help you visualize your evaluation concepts – research design – impact – program logic charts

IPDET

54 54

Experimental Research Design Program Group

Comparison Group

Baseline Time + n

IPDET

55 55

Quasi-Experimental Design Program Group

Comparison Group

Baseline Time + n

IPDET

56 56

Historical or Retrospective Design Program Group Baseline Time + n

IPDET

57 57

Impact • You may be able to illustrate impact using a graphic • Compare the measurement at baseline to the results

IPDET

58 58

Data to Illustrate Impacts of MICAH 60 50 40 30 20 10 0

Baseline (1996)

gh t

g

w ei

tin U nd er

Source: Fabricated data, 2006

W as

nt in g

Follow-up (2000)

St u

Percent of Population

Example Impact Graphic

IPDET

59 59

Theory of Change Model • Show the model used for theory of change – – – –

program theory model logic model outcome model logical framework

• Should have been validated by the program “owners” IPDET

60 60

Too Much of a Good Thing

IPDET

61 61

Oral Presentations: Planning for Your Audience • Ask these questions: – Who is your audience? – What do they expect? – What is your point? • What are the three things you want the audience to remember?

– How much time do you have? – What are the resources of the room for delivery: slides, overheads, PowerPoint, posters? IPDET

62 62

Preparing Your Presentation • Always keep your audience in mind • Simple rule for presentations: – Tell them what you will tell them – Tell them – Tell them what you told them

IPDET

63 63

Enhancing Your Presentation • Make visual enhancements to help communicate • Consider making copies of some data or visuals for audience • Have a back-up plan in case of electricity or equipment failure (overhead projector, computer, etc.) • Have a few well-chosen handouts IPDET

64 64

Designing Overheads and Handouts • Use few words • Use clear visuals • Use lots of white space – norm is no more than eight lines of text for a single slide

• Audience is there to listen, you do not need to put everything into overheads and handouts (continued on next slide)

IPDET

65 65

Designing Overheads and Handouts (cont.) • Handouts before or after presentation? – People tend to look ahead, you may loose attention – If you are presenting complex data or tables, hand out the tables as you talk about them

• If printing slides, no more than two slides per page IPDET

66 66

Practicing Your Presentation • Rehearse alone at first • Then, rehearse in front of another person or persons • Get feedback from others • Adjust your presentation based on what you feel and what others say • As you practice, keep track of time • Talk to people, not your notes, try to make eye-contact IPDET

67 67

Peer Review • Process used for checking the work performed by one’s equals (peers) • Peers evaluate work based on specific criteria

IPDET

68 68

Meta-Evaluation • Evaluations of an evaluation • Done to identify strengths, limitations, and/or other uses of evaluations

IPDET

69 69

Meta-Evaluation Criteria of Merit • • • • •

Validity Utility Credibility Cost-effectiveness Ethicality

IPDET

70 70

Five Ways to Do MetaEvaluation • Apply the KEC list to the evaluation itself • Use a special meta-evaluation checklist • Replicate the evaluation, doing it the same way, and compare results • Do the evaluation using a different methodology and compare the results • Apply the Program Evaluation Standards IPDET

71 71

To continue on to the Next Module click here To return to the Table of Contents click here