Introductory Physics Students’ Treatment Of Measurement Uncertainty

Introductory Physics Students’ Treatment (or ignoring) of Measurement Uncertainty Duane L. Deardorff Robert J. Beichner - advisor The University of North Carolina at Chapel Hill

Presented at the 123rd National Meeting of the American Association of Physics Teachers in Rochester, NY Monday, July 23, 2001

www.physics.unc.edu/~deardorf/uncertainty 1

Are these time measurements significantly different? t1 = 1.86 s t2 = 2.07 s • Yes • No • Can’t tell 2

Are these time measurements significantly different? t1 = 1.86 s t2 = 2.07 s Student responses (N = 44): • Yes 43% • No 27% ??? • Can’t tell 30% 3

Motivation • Science relies on empirical data, which is inherently subject to measurement error • Uncertainty estimates are necessary for: – assessing quality of data – comparison of data – verify/refute theoretical predictions

• Students often have difficulty analyzing errors • Guidelines for reporting uncertainties vary: – Terminology and notation is not consistent – International standard exists but is not well known 4

Primary Research Questions • How do physics students treat uncertainty? • What do and don’t introductory physics students understand about measurement uncertainty? • How do these student conceptions compare with those of experts and other students?

5

Measurement Uncertainty Studies • M. Sere, R. Journeaux, C. Larcher (1993) 20 first-year physics lab students in France • Students did not understand the role and value of statistical tools for specifying confidence • Confusion between systematic and random errors; precision and accuracy

• S. Allie, A. Buffler, L. Kaunda, B. Campbell, F. Lubben (1998) 121 first-year physics students at U. of Cape Town, South Africa • The need to take additional measurements depends on the context of the experiment, but student rationale is not clear.

• J. Tomlinson, P. Dyson, J. Garratt (2001) 33 first-year chemistry students at Univ. of York, UK • Less than half demonstrated a “good” understanding of 12 key terms related to measurement error. 6

Research Methodology • Formative study based on grounded theory • Research methods are primarily qualitative with some quantitative comparisons • Student ideas examined from a variety of perspectives to triangulate common patterns: – – – – –

Open-ended surveys: N ≈ 200 Follow-up student interviews: N ≈ 50 Homework and class assignments: N ≈ 300 Physics lab reports: N ≈ 100 Hands-on lab exam: N ≈ 200 7

Population and Sampling Population: introductory physics students – calculus and algebra-based levels, sem. 1 & 2

Student samples: – North Carolina State University (NCSU) • Almost no error analysis

– University of North Carolina (UNC-CH) • Heavy emphasis on error analysis

Experts: graduate students and teachers – Surveyed to determine the “right” answers 8

Research Findings 1) Uncertainty is rarely estimated and stated, even when required (true for experts too!). 2) Even if found, most students do not use uncertainties to justify their conclusions. 3) Calculated values are often reported with too many (in)significant digits. 4) Students have difficulty identifying the primary source of error in an experiment. 9

Research Findings 1) Students often fail to report a quantitative uncertainty estimate, even when requested. Overall reporting rates from this study: 0 to 50% of students reported uncertainty 30 to 70% of TAs reported uncertainty

10

Research Findings Task: Use a ruler to measure the diameter of a penny as accurately as possible. Uncertainty Reported for Penny Dia. 100% 80% 60% 40% 20% 0%

NCSU UNC

Students

TAs 11

Research Findings 2) Even if stated, most students do not justify their conclusions based on the uncertainty Judgements are made based on arbitrary criteria: – “Our percent error was only 4%, so our experiment proved the theory.” – “I decide by how much two measurements differ in order to see if they agree.” – “The result should be accurate as long as the error is less than 10%.”

Similar findings in other studies (Sere; Garratt)

12

Research Findings 3) Students tend to overstate precision (too many significant figures) of calculated values and to a lesser extent for directly measured values. Typical student values: L W

L = 2.35 cm (± 0.05 cm) W = 1.85 cm (± 0.05 cm) A = L*W = 4.3475 cm2 (Expert: A = 4.3 ± 0.1 cm2 ) 13

Research Findings 4) Students have difficulty identifying the primary source of error in an experiment. Are nickel coins made of nickel? Find density: • Median density from 76 students = 7.1 g/cm3 (standard deviation = 10 g/cm3 ) Density of pure nickel = 8.912 g/cm3 Density of nickel coin = 8.9 ± 0.4 g/cm3 (alloy of 25% nickel, 75% copper)

14

Sources of Error for Nickel Density Actual Source of uncert. error contrib. thickness ±10% διαµετερ ± 1% µασσ ηυµαν οτηερ

± 2%

NCSU UNC students students TAs (n = 36) (n = 40) (n = 17) 15% 42% 40% 19%

16%

20%

41%

16%

40%

7% 18%

3% 23% 15

Conclusion Students often focus on the details of error analysis and miss the big picture, losing sight of the forest for the sake of the trees.

16

Student:

t1 (s) 1.86 1.74 2.15 Average = 1.92

Expert:

t2 (s) 2.07 1.89 2.20 2.05

Std. Dev. = Std. Error =

t1 (s) t2 (s) 0.21 0.16 0.12 0.09

“The numbers are close, but different.”

t1 t2 1.8

2.0

2.2

t (s)

“These time measurements agree with each other.” 17

Implications for Instruction • Structure lab activities to intrinsically motivate students to estimate uncertainties • Remind students of the “big picture” view of why uncertainty estimates are important. • Show examples of how to decide whether results agree or disagree within their uncertainty using error bars on number line. • Teach concepts consistent with ISO Guide

18

ISO Guide to the Expression of Uncertainty in Measurement (GUM) • International Organization for Standardization published new guidelines in 1993 for industry and research

NIST version: physics.nist.gov/cuu/Uncertainty • Use standard uncertainty – Type A component: random, evaluated statistically – Type B component: systematic, judgement, a priori

• use term “uncertainty” not “error” • avoid use of ambiguous ± notation

19

Future Research and Development • • • • •

Refine facets of understanding (now on Web) Extend research using Lab Exam Develop concept test on meas. uncertainty Create new teaching materials Distribute data comparison tool (Excel)

www.physics.unc.edu/~deardorf/uncertainty 20

“It is better to be roughly right

than precisely wrong.” - Alan Greenspan U.S. Federal Reserve Chairman For more on measurement uncertainty, go to:

www.physics.unc.edu/~deardorf/uncertainty 21