Gum Guide

Australasian Physical & Engineering Sciences in Medicine Volume 28 Number 2, 2005

EDUCATIONAL NOTE

A standard approach to measurement uncertainties for scientists and engineers in medicine* K. Gregory1,2, G. Bibbo1,3 and J. E. Pattison1

1

School of Electrical and Information Engineering (Applied Physics), University of South Australia, Mawson Lakes, Australia 2 Radiation Protection Division, Environment Protection Authority, Kent Town, Australia 3 Division of Medical Imaging, Women’s and Children’s Hospital, North Adelaide, Australia

Abstract The critical nature of health care demands high performance levels from medical equipment. To ensure these performance levels are maintained, medical physicists and biomedical engineers conduct a range of measurements on equipment during acceptance testing and on-going quality assurance programs. Wherever there are measurements, there are measurement uncertainties with potential conflicts between the measurements made by installers, owners and occasionally regulators. Prior to 1993, various methods were used to calculate and report measurement uncertainties. In 1993, the International Organization for Standardization published the Guide to the Expression of Uncertainty in Measurement (GUM). The document was jointly published with six international organizations principally involved in measurements and standards. The GUM is regarded as an international benchmark on how measurement uncertainty should be calculated and reported. Despite the critical nature of these measurements, there has not been widespread use of the GUM by medical physicists and biomedical engineers. This may be due to the complexity of the GUM. Some organisations have published guidance on the GUM tailored to specific measurement disciplines. This paper presents the philosophy behind the GUM, and demonstrates, with a medical physics measurement example, how the GUM recommends uncertainties be calculated and reported.

(eg thermometers and blood analysers), also need to maintain high performance levels. There are examples in the literature showing that the health care industry makes widespread use of quality assurance (QA) programs3-5 to monitor performance of equipment. These QA programs usually include a variety of measurements that are conducted at regular intervals. Performance measurements may also be conducted on new equipment as part of acceptance testing. As medical equipment is subject to so many measurements, there is potential for conflicts to arise between the various stakeholders, such as installers, operators, purchasers, manufacturers and regulators. Any measurements involved in such conflicts are likely to be closely scrutinised, and the question of measurement uncertainty may arise. Prior to 1993, there was no international consensus on how measurement uncertainties were to be calculated and reported6. The variation in philosophies and calculation methods caused three main problems, especially for laboratories conducting comparisons with other laboratories in different countries6. The first problem was that laboratories had to give extensive explanations of the measurement uncertainty calculation method they used whenever results were reported, so that other laboratories could make comparisons. The second problem was that it

Key words uncertainty, error, accuracy, GUM, measurement

Introduction The performance tolerances of medical equipment are generally more stringent than domestic and industrial equipment due to the critical nature of health care. In the case of medical X-ray units where patients are exposed to ionising radiation, the performance levels may be set by legal instruments, such as is done in South Australia1, or by published recommendations2. Other medical equipment, such as those relating to the administration of drugs (eg balances and flow rate meters), and patient monitoring *Presented in part at the 29th annual conference of the Australasian Radiation Protection Society, Adelaide, SA, Australia, 24-27 October 2004. Corresponding author: Kent Gregory, Radiation Protection Division - EPA, PO Box 721, Kent Town, SA 5071, Australia Tel: 08 8130 0713, Fax: 08 8130 0777 Email: [email protected] Received: 14 February 2005; Accepted: 14 March 2005 Copyright © 2005 ACPSEM/EA

131

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

was common to keep random and systematic uncertainties separated, both during calculations and in the final reporting, even though they were both uncertainties. The final problem was that some uncertainties were deliberately overstated in the belief that this was acting conservatively. Such practise has the capacity to increase type I and type II errors7 in hypothesis testing (a type I error is when a good result is erroneously rejected, while a type II error is when a poor result is erroneously accepted). A desire to overcome these three problems was the driver for what eventually became the Guide to the Expression of Uncertainty in Measurement6 (GUM). In 1993, the GUM was jointly published by the International Organization for Standardization and six international organisationsa primarily concerned with measurements and standards. The document is therefore recognised by organisations internationally, including the Australian National Measurement Institute (NMI). NMI holds most of Australia’s First Level Standards, with the exception of those relating to ionising radiation, which NMI has delegated to the Australian Nuclear Science and Technology Organisation and the Australian Radiation Protection and Nuclear Safety Agency. The GUM is underpinned with an extensive mathematical basis and has the capacity to deal with most measurement uncertainty problems. This generalist approach comes at a cost; the GUM concepts and methodologies are non-trivial, and may be difficult to comprehend without continuous involvement in measurement science at the highest levels (e.g. working at institutions such as NMI). Some organisations8-10 have published specific guidance on the GUM for various measurement disciplines. Others11,12 have published explanatory texts on the GUM in general. In particular, the National Measurement Laboratory (part of NMI) has published a document12 (the NMI Guide), which explains in simple terms the philosophy, methods and reporting style of the GUM for use in applied environments. The NMI Guide also offers a number of pragmatic approaches that are in keeping with the philosophy of the GUM, but substantially reduce the complexity of the calculations. The NMI Guide is still somewhat complex and requires some introduction and further explanation before engineers and scientists adopt the GUM more widely. Indeed, there appears to be very little application of the GUM by scientists and engineers in medicine within published literature. These workers may be making use of simpler methods13,14 for dealing with uncertainties that were introduced to them during their undergraduate university courses. This paper presents a simplified theoretical background, and uses a medical physics example to illustrate how to apply the GUM to measurements made by scientists and engineers in medicine.

Theoretical background Review of statistics All measurements have an uncertainty. If a large number of measurements are taken and each measurement value is plotted against the number of times a measurement value occurs (i.e. a histogram), the end product would be the measurement’s uncertainty distribution. Uncertainty distributions are typically scaled (standardised) so that the area under a curve defining the histogram is equal to 1. This is achieved by changing the x-axis to represent the difference between each measurement value and the mean value in terms of the number of standard deviations, and changing the y-axis to represent probability density. Thus, the probability of a measurement lying between two values is equal to the area under the curve between these two points. Most scientists and engineers will be familiar with the Normal distribution, also known as the Gaussian distribution and depicted in figure 1(a), which has a bellshaped curve. The Normal distribution is commonly encountered in many types of measurements. The reason why the Normal distribution is so common is explained by the Central Limit Theorem. The ubiquitous nature of Normal distributions is utilised by the GUM, and is discussed later. Another uncertainty distribution is the rectangular distribution as depicted in figure 1(b), which applies to measurements that are rounded or truncated (such as by a digital display). Occasionally, some measurements are encountered that have uncertainty distributions that are neither Normal nor rectangular, such as those depicted in figures 1(c) and 1(d). The aim of uncertainty analysis is to estimate the uncertainty distribution for a final measurement result. The final result’s uncertainty distribution is entirely dependent on the uncertainty distributions of all factors that influence

Figure 1. Uncertainty distributions are like frequency histograms, except they are standardised so that the area under an uncertainty distribution curve is equal to 1. This is achieved by changing the x-axis to represent the difference between measurement values and the mean in terms of the number of standard deviations (instead of measurement values), and changing the y-axis to represent probability density (instead of frequency). Uncertainty distributions can take any form, but the most common is the Normal distribution (a). The rectangular distribution (b) is associated with rounding or truncation of readings. Any uncertainty distribution, including (c) and (d), may be assigned to an uncertainty component.

a

International Bureau of Weights and Measures, International Electrotechnical Commission, International Federation of Clinical Chemistry, International Union of Pure and Applied Chemistry, International Union of Pure and Applied Physics, and the International Organization of Legal Metrology.

132

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

the final result. This may include other measured quantities (e.g. the measurement of mass and volume to determine density), rounding of readings, or variations in atmospheric or other conditions. Note that all factors that influence the final result are collectively referred to as uncertainty components (a term derived from the NMI Guide). Thus a major task in uncertainty analysis is to characterise the uncertainty distributions of the uncertainty components, so that they can be combined. In order to characterise an uncertainty distribution exactly, the entire population of measurements would need to be plotted as a histogram. As this is not possible for infinite populations of measurements, a finite number of measurements are sampled instead. From this sample, inferences can be made about the infinite population. In the case of the common Normal distribution of measurements, the distribution of a finite sample of such measurements is represented by a t-distribution (also called the Student’s t-distribution). A t-distribution is virtually identical to a Normal distribution when a sufficiently large number of measurements are sampled (such as more than 20 measurements). However, if fewer measurements are sampled, the shape of the t-distribution is flatter and broader (see figure 2). This shape change reflects the fact that with fewer measurements, less is known about the infinite population. The shape of the t-distribution is dependent on the degrees of freedom, ν. The value ν equals the number of measurements sampled minus the number of quantities calculated from the measurements. In most cases, the measurements are used to calculate a mean, so ν will equal one less than the number of measurements sampled. For rectangular distributions, measurement sampling is usually not required to ascertain the distribution shape. Instead, its shape is obtained intuitively; the distribution is flat with upper and lower bounds defined by, say, the resolution of the instrument.

The uncertainty distribution of the final result is in most cases, a known shape, due to the Central Limit Theorem (CLT). The CLT states that when uncertainty distributions are combined, the resulting distribution approximates a Normal distribution. Knowing the shape of the final result’s uncertainty distribution reduces the amount of information needed regarding the distributions of all the uncertainty components. In fact, only three quantities for each uncertainty component are needed by the GUM to estimate the uncertainty distribution of the final result. The first quantity used to describe an uncertainty distribution is the standard uncertainty, u (or estimated standard deviation). This is a measure of the spread of the distribution, with small u values indicative of repeatable measurements (i.e. close agreement of successive measurements). The second is the degrees of freedom, ν. This quantity reflects the reliability of the value u. For example, a value for u derived from many measurements should have greater influence on the final result than another u derived from a smaller number of measurements. Thus, the higher the value of ν, the more reliable the value of u. The third quantity is the sensitivity coefficient, c. Values for c are used to take into account the relative sensitivity of the final result to small changes in uncertainty components. Thus if c = 1, an increase or decrease in the value of an uncertainty component causes an equivalent change in the final result. When c > 1 or c < 1, the uncertainty component has, respectively, a greater and lesser effect on the final result. Background to the GUM and NMI Guide The GUM and NMI Guide use methods and terminology that may be unfamiliar to users of other uncertainty analysis methodologies. For example, the term measurand, which describes the quantity or parameter subject to measurement, is used extensively in the GUM and NMI Guide. While a detailed justification of why certain procedures are used is documented in the GUM, together with derivations of the equations, a brief background is provided here, together with essential terminology. It is important to note that the GUM is not completely regimented; there remain steps in the process requiring professional judgment. As such, uncertainty analysis remains an inexact science, since professional judgment is subjective. This leads to uncertainty in the uncertainty estimates. The NMI Guide plays on this fact, and proposes a number of pragmatic approaches to the GUM process that theoretically increase the uncertainty in the uncertainty estimate. However, the NMI Guide argues that in practice these increases are insignificant compared to the uncertainty introduced through professional judgment. There are four main steps in the GUM process, and these are described in more detail below.

Figure 2. A comparison of the Normal distribution and tdistributions with 1, 3 and 10 degrees of freedom. When the degrees of freedom equals 20 or more, the t-distribution and the Normal distribution may be considered equivalent for the purposes of calculating uncertainties.

Step 1: Modelling the measurement The first step in the process is to make what the GUM refers to as a model of the measurement. This is a simple task that involves writing an equation that relates the 133

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

measurements cannot be obtained (e.g. a single value from a table of reference data). With only a single value, statistics cannot be used to calculate u and ν. Hence, u and ν are derived using non-statistical approaches (i.e. professional judgment), some of which have been simplified by the NMI Guide. The first step in deriving u and ν in a Type B evaluation is to assign an uncertainty distribution representative of how the value of the uncertainty component might vary if more values were available. The GUM proposes that one of the four distributions in figure 1 be assigned, with the triangular and trapezoidal distributions assigned when there is insufficient confidence to suggest the data do not follow a Normal distribution, but sufficient information to suggest more certainty than a rectangular distribution. The NMI Guide suggests that uncertainty components, such as rounding or truncation of readings, be assigned a rectangular distribution, while almost all others may be assigned a Normal distribution. The NMI Guide suggests that triangular and trapezoidal distributions are rarely encountered in the real world, and that more information should be obtained before assigning such uncertainty distributions to any measurement. Having assigned an uncertainty distribution, the next step is to estimate the range, ±a, (figure 3a and 3b) within which the true value lies, and the likelihood that the true value is within this range. If a rectangular distribution was assigned, a value for a can be chosen such that the range ±a represents exactly 100% of the uncertainty distribution. The standard uncertainty can then be calculated;

(a)

(b) Figure 3. (a) For a rectangular uncertainty distribution, the value a is selected such that the range ±a includes the true value with 100% confidence; (b) For the Normal distribution, the value a is selected such that the range ±a includes the true value with approximately 95% probability. The estimated standard uncertainty is half the value of a. If a very conservative value is chosen (a′ ) such that the range ±a′ almost certainly includes the true value (nearly 100%), half the value of a′ will be a poor estimate of the standard uncertainty.

u=

a for a rectangular distribution (figure 3a) 3

(1)

If a Normal distribution was assigned, the NMI Guide suggests choosing a value for a such that the range ±a represents 95% of the measurements. The standard uncertainty can then be calculated;

various uncertainty components to the final result (see equation 9 in the example). Modelling the measurement serves two purposes; it helps to identify uncertainty components, and it assists in finding values of c for the uncertainty components.

u=

a for a Normal distribution (see figure 3b) 2

(2)

It is important not to deliberately increase a beyond what is necessary, such as a′ in figure 3b. By deliberately inflating the value a, the uncertainty calculated for the final result may be unnecessarily large. In order to assign values of ν, the NMI Guide argues that the final result is not particularly sensitive to the ν values chosen. If that is the case, then it is sufficient to categorise each evaluation broadly as either poor, reasonable, good or excellent (see table 1), and to assign the corresponding value for ν. (Note that the third column in table 1, k, is explained in Step 4.) Values of the sensitivity coefficient c may be obtained in one of three ways; (i) by making a small change in the value of the uncertainty component in the model of the measurement and calculating the consequent change in the final result, (ii) experimentally changing the value of the uncertainty component slightly and observing the consequent change in the final result, or (iii) using calculus

Step 2: Calculating values for u, ν and c The GUM process involves characterising all uncertainty components with these 3 descriptors; u, ν and c. Values for the standard uncertainty u and the degrees of freedom ν are calculated or assigned following what the GUM refers to as either a Type A or Type B evaluation. It may be helpful to think of Type A and Type B evaluations as those uncertainties commonly described as ‘random’13,14 and ‘systematic’13,14 respectively (although this comparison is not strictly true). In Type A evaluations, the value u is equal to the estimated standard deviation of the population of measurements, while ν is equal to its degrees of freedom. Type B evaluations are performed when Type A evaluations cannot be performed, such as when there is only one value for an uncertainty component and additional 134

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Table 1. Relationship between measurement quality (as judged through experience), degrees of freedom (ν) and coverage factor (k) to generate a 95% confidence interval.

Measurement quality

ν

k for 95% confidence interval

Poor Reasonable Good Excellent

2 8 20 1000

4.30 2.31 2.09 1.96

Note that equation (5) is simply the linear sum of the weighted standard uncertainties of the uncertainty components. If any uncertainty components happen to be correlated, it may be possible to alter the measurement procedure to obtain uncorrelated uncertainty components, and thus retain the simpler equation (3). However, correlation may be a desirable effect, especially since anticorrelated uncertainties can reduce the combined standard uncertainty. The NMI Guide provides a pragmatic approach to correlations to reduce the mathematical process. With regard to condition (b), equations (3) and (4) are the first terms in a Taylor series expansion, and are good approximations if an approximately linear relationship exists between the final result and each of the uncertainty components. If this is not the case, the next term in the Taylor series will need to be added to equations (3) and (4);

to differentiate the model of the measurement with respect to the uncertainty component (this is a common uncertainty analysis technique14). While it may not be possible to use all three methods to calculate each value of c, at least one method will be.

∑

N

i =1

where ui is the ith standard uncertainty, and ci is the ith sensitivity coefficient. Equation (3) is valid if two conditions are met; (a), each uncertainty component is not correlated with any other uncertainty component, and (b), an approximately linear relationship exists between each uncertainty component and the final result for the range of values the uncertainty components are likely to have. For most measurements, if both the above conditions are not met, it is usually possible to alter the way the measurement is made so that both conditions are satisfied. If not, the mathematics becomes more complex. With regard to condition (a), the degree of correlation is determined by calculating the correlation coefficient, which can have any value from –1 (anti-correlated) to +1 (fully correlated). Values close to 0 are indicative of uncorrelated uncertainty components. To account for correlation, equation (3) becomes: u 2 comb = ∑i =1 (ci ui ) + 2∑i =1 N

2

N −1

∑

N j =1+i

ci c j ui u j r (xi x j )

u comb = ∑ (ci ui )

j =1

(6)

the final result’s uncertainty distribution can be treated mathematically like a t-distribution. In practice, νeff will typically have a value greater than 20. Step 4: State the final result The final result’s uncertainty may be expressed simply as ±ucomb, which represents a range of values within which the true value is expected to lie with approximately 68% probability (assuming νeff≥20). This is the traditional one standard deviation or one sigma level. However, the GUM reports the expanded uncertainty, U, calculated using equation (8);

(4)

where r(xixj) is the correlation coefficient of the two uncertainty components xi and xj. In the special case where all uncertainty components are fully correlated, i.e. r(xixj) =1, equation (4) reduces to; N

i =1

 1  ∂ 2 f  2 ∂f ∂ 3 f   + ui2u 2j    2  ∂xi ∂x j  ∂xi ∂xi ∂x 2j   

where f is the model of the measurement with N uncertainty components. Non-linearity may occur when the value of the final result is very small compared to the value of the uncertainty components (such as calculating the area of a small rectangle by measuring two sides using a ruler with a large uncertainty), or the uncertainty components vary the final result in a non-linear manner (such as an uncertainty component raised to a power in the model of the measurement). The GUM provides more detail on nonlinearity. As mentioned earlier, the CLT suggests that the final result’s uncertainty distribution is a Normal distribution, but this is only an approximation. However, with the combined standard uncertainty, ucomb, and the effective degrees of freedom, νeff, obtained from the WelchSatterthwaite formula6; 4 ucomb (7) ν eff = 4   ( c u ) N ∑i=1  νi i   i 

(3)

(c i u i ) 2

N

∑∑

Step 3: Calculation of ucomb and νeff Using the u, ν and c values, equations are then used to calculate the combined standard uncertainty, ucomb and the effective degrees of freedom of the final result, νeff. Given two conditions specified below, the equation to calculate ucomb is the square root of the sum of the squares of the standard uncertainties, weighted with the respective sensitivity coefficients. Thus for a final result with N uncertainty components, the combined standard uncertainty of the final result is given by: u comb =

N

U = ucomb×k

(8)

where U is the expanded uncertainty, and k is the coverage factor.

(5)

i =1

135

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

Table 2. An uncertainty analysis table. Each row represents a different uncertainty component, with values for standard uncertainty (u), degrees of freedom (ν) and sensitivity factor (c).

Uncertainty components

u

ν

c

Reading Rounding Cal. Factor Temperature Pressure Position (vertical plane) Position (anode-cathode plane)

0.0127 mGy 0.000289 mGy 0.025 units 0.50 K 50 Pa 2 mm 2 mm

24 1000 20 20 20 20 20

1.017 mGy/mGy 1.017 mGy/mGy 1.953 mGy/unit 0.00668 mGy/K -0.0000196 mGy/Pa 0.00592 mGy/mm 0.00272 mGy/mm

Position (lateral plane)

2 mm

20

0 mGy/mm

ucomb

= 0.0523 mGy

The combined standard uncertainty and effective degrees of freedom: Values for U can be calculated such that ±U has any desired probability of containing the true value, including the increasingly common12 95% probability. The value k is selected from tabulated data6,12, given νeff and the desired probability (known as confidence interval, CI). Some typical values of k appear in table 1. Both the final result and U need to be rounded such that three objectives are met; namely U contains only significant digits, the final result and U are rounded to the same least significant digit, and the rounding process itself does not introduce a significant uncertainty. Rounding is described in more detail in the NMI Guide, but the basic steps appear below.

Step 3:

Step 4:

Calculate the values of ucomb and νeff. For N uncorrelated uncertainty components, the values of ucomb and νeff are given by equations (3) and (7), respectively. State the final result. The statement should mention that the uncertainty was calculated in accordance with the GUM, give the values of U and k, indicate the CI used, and ensure the final result and U are rounded appropriately.

Example Consider the measurement of radiation dose from a mammographic X-ray unit with exposure settings of 28 kVp, 20 mAs, with large focus selected, molybdenum anode, 30 µm molybdenum filtration, at a position 40 mm from the chest wall in the anode-cathode plane, 45 mm above the imaging plate in the vertical plane, and centred on the remaining axis (i.e. centred laterally with respect to the patient). All data for the example are summarised in table 2.

1. Round the final result and U once only. 2. Round U to 1 significant digit if the first significant digit is 5 or more, otherwise round to 2 significant digits. (eg 0.0078 becomes 0.008, while 0.013 is unchanged). 3. Round U upwards unless rounding down will only change U by a few percent (eg 0.0078 becomes 0.008, while 0.0071 becomes 0.007) 4. Round the final result to match the significant digits in U. 5. Round a 5 in the final result to the nearest even number of the next significant digit (eg 10.65 becomes 10.6, while 10.75 becomes 10.8).

Step 1. Make a model of the measurement Suppose that a calibrated, vented ionisation chamber is used to measure the dose. The temperature and pressure of the ambient atmosphere are also measured and used to compensate the ionisation chamber. Given this information, the measurement model is:

Summary of the GUM method The GUM process is to; Step 1: Make a model of the measurement. For measurements with many uncertainty components, it may become difficult to manage the model. In such cases, it may be appropriate to separate the model into smaller parts, and analyse these separately. Step 2: Identify and characterise each uncertainty component. The NMI Guide suggests creating an uncertainty analysis table for recording the results of this step, such as table 2.

 temperature  final result = reading × cal. factor ×  ×  295.15 

(9)

 101325   pressure   

where: final result is the dose to be determined, reading is the value displayed by the electrometer, cal. factor is the calibration factor for the ionisation chamber and electrometer (a unitless value), temperature is the temperature inside the ionisation chamber at the time of 136

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

measurement (in K), pressure is the pressure inside the ionisation chamber at the time of measurement (in Pa). The final result is 1.97254 mGy, based on a reading of 1.940 mGy, a calibration factor of 1.01, pressure of 100 650 Pa, and temperature of 295.15 K. Step 2. Identify and characterise each uncertainty component Four uncertainty components were identified when modelling the measurement and appear in equation 9. Four more uncertainty components have been identified; rounding of the reading by the display, and variation of the chamber’s position from the desired position in the three spatial planes. Calculation of c For the purpose of demonstration, values of c have been acquired using the three different methods for three groups of uncertainty components. (i) For the reading, the rounding of the displayed reading, and the calibration factor, the differentiation method is used. Differentiating the model with respect to ‘reading’: c=

∂ ( result )  temperatur e   101325  = cal. factor ×  ×  ∂ ( reading )  295.15   pressure 

c = 1.017 calculated using measured values for the other uncertainty components. For rounding, as the rounding of the reading is essentially a change in the value of the reading, again, c = 1.017. For the calibration factor, c=

∆result 0.00668 = = 0.00668 mGy/K ∆temperature 1

Similarly, increasing the pressure by 100 Pa to 100750 Pa decreases the final result by 0.00196 mGy to 1.97058 mGy. Thus c=

Calculation of u and ν Both the Type A and Type B evaluation methods will be demonstrated in the evaluation of u and ν for each uncertainty component. (i) When the reading value was recorded, a further 24 individual readings were also taken to obtain information about the distribution of these values. Thus, a Type A evaluation can be used. The standard uncertainty of the reading is equal to the estimated standard deviation of the 25 readings, i.e. 0.0127 mGy. The value ν in this case equals one less than the number of data used. Thus ν = 24. (ii) The rounding of a displayed reading could be any value in the range ±0.0005 mGy, as the reading is displayed to 3 decimal places. Furthermore, there is equal probability of the rounding being any value in this range, and zero probability of the rounding being outside the range. This is an example of a rectangular distribution. Thus from equation 1;

∂(result)  temperature   101325  = reading ×  ×  ∂(cal. factor)  295.15   pressure 

c = 1.953 mGy/unit (ii) For the temperature and pressure, values for c are derived by changing the value of these uncertainty components by a small amount in the model equation (9), and observing the change in the final result. Increasing the temperature by 1 K to 296.15 K increases the final result by 0.00668 mGy to 1.97922 mGy. Thus c=

inverse square law is simple, modelling exposure variation in the other planes is non-trivial, which makes these uncertainty components well suited to an empirical approach to evaluate c. Vertically, a 10 mm position change altered the final result by 0.0592 mGy. Therefore c = 0.00592 mGy/mm. In the anode-cathode plane, a 10mm position change altered the final result by 0.0272 mGy. Therefore c = 0.00272 mGy/mm. Laterally, a 10 mm position change did not alter the final result. Therefore c = 0 mGy/mm.

∆result − 0.00196 = = −0.0000196 mGy/Pa ∆pressure 100

(iii) For positioning, values for c were obtained using the empirical method. Dose varies with position in the 3 spatial planes; vertically (inverse square law), the anode-cathode plane (‘heel’ effect) and laterally. Note that while modelling the

u=

a 0.0005 = 0.000289 mGy = 3 3

As the boundaries of the distribution are precisely known, this evaluation would be judged as ‘excellent’ in accordance with table 1, thus ν = 1000. Assume the calibration factor was provided with only a limited explanation, such as “…the calibration factor = 1.01, with an uncertainty of 5%…”. To evaluate u and ν, a number of assumptions will need to be made. The first assumption is that the uncertainty distribution of the calibration factor is a t-distribution. This is a reasonable assumption as it is consistent with the CLT. The next assumption is that it would be expected that coming from a laboratory with traceable standards, the calibration was at least ‘good’. This translates to ν=20 from table 1. Finally, a decision needs to be made with regard to what confidence interval was used. If the stated uncertainty of 5% was assumed to represent a 95%CI, then from equation 2; u=

a 5% = = 2.5% = 0.025 . 2 2

(iii) Atmospheric conditions were measured using a portable temperature and pressure meter. The meter’s specifications stated that temperature measurements were accurate to ±1°C, and pressure measurements were accurate 137

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

to ±100 Pa. To gain a greater insight into the uncertainty of temperature and pressure readings from the meter, comparisons were made at ambient temperature and pressure with a calibrated thermometer (±0.2°C, 95%CI) and a calibrated barometer (±50 Pa, 95% CI). For all measurements taken over a period of several weeks, the difference between the portable meter and the calibrated instruments was less than the stated ‘accuracy’. It is therefore reasonable to assume that uncertainties of ±1°C and ±100 Pa represent 95%CIs, and that the evaluation for both is at least ‘good’. Hence, from equation 2, u for temperature = 0.50 K, u for pressure = 50 Pa, and from table 1, ν = 20 for temperature and pressure. (iv) The uncertainty in placing the ionisation chamber in the specified location of the three spatial planes was assessed by repeatedly placing the chamber in position for measurement and measuring its actual position with digital vernier calipers. The measurement with calipers is not done routinely; it is too time consuming. However, the data regarding positioning uncertainty enabled values for u and ν to be estimated for the three positioning uncertainty components. It was estimated that for all three planes, the ionisation chamber was within a = 4 mm of the correct location 95% of the time, and the measurements followed a Normal distribution. From equation 2, u = 2 mm. The evaluation was regarded as good (ν = 20). Step 3. Combine the uncertainties This step is most simply performed using a spreadsheet program. Using equations (3) and (7) gives, ucomb = 0.0523 mGy, and νeff = 26. Step 4. State the final result. Using tabulated data6,12, the appropriate coverage factor, k, for the desired confidence interval can be found. If a 95%CI is chosen, and given νeff = 26, then k = 2.056. Hence U = 0.1074 mGy for a 95%CI. For Excel spreadsheet users, U can be calculated using the formula ‘=TINV(0.05,ROUND(νeff,0))× ucomb’. Given the final result of 1.97254 mGy, the final result and uncertainty need to be rounded so they are harmonised. Firstly, round U in accordance with the rounding method, yielding ±0.11. Next, round the final result to the same decimal place, yielding 1.97 mGy. The following statement can now be made: “Under the conditions prescribed for the measurement the exposure was found to be 1.97 mGy. The uncertainty was calculated in accordance with the ISO GUM, and was found to have a 95% confidence interval of ±0.11 mGy and 26 degrees of freedom.” The uncertainty analysis data for this example is shown in table 2.

Discussion A detailed description of the quantity to be measured is an important step in reducing measurement uncertainty.

The omission of any detail, such as whether or not the X-ray unit should be warmed up prior to testing, can cause different (yet valid) final results. Note that for simplicity of the example, the final result was based on the value of one individual reading, while a further 24 readings were taken but only used to estimate the standard deviation of the reading. However, it would be more sensible to use the mean of the 25 individual readings as the value of ‘reading’. If this were done, the standard uncertainty of ‘reading’ would take the smaller value of the experimental standard deviation of the mean (ESDM)6,11,12. Since the ESDM equals the standard deviation of the individual data divided by the square root of the number of readings, the standard uncertainty of ‘reading’ would be five times smaller. Also note from the example that equation 3 was used, despite one of the uncertainty components (vertical position of the ionisation chamber) varying with the final result in accordance with the inverse square law, not linearly as required by equation 3. However, given the measurement was conducted approximately 600mm from the anode, and that the standard uncertainty was 2 mm, the variation of vertical position and dose is approximately linear over this range. From the data in the uncertainty analysis table, table 2, an extra column for the result c2u2 for each uncertainty component can be generated. Uncertainty components with larger values of c2u2 contribute the largest proportion to the overall measurement uncertainty, and so effort may be directed to improving these measurements. Conversely, there is little to be gained by directing effort towards improving the measurements of uncertainty components with low values for c2u2. It is only necessary to include uncertainty components that are significant in the uncertainty analysis table. If it is unknown as to whether or not an uncertainty component is significant, it should be included.

Conclusions Those who are familiar with other measurement uncertainty methods, especially in the field of radiotherapy15, may recognise elements of the GUM process. The advantage of using the GUM process is that it provides a uniform approach to the determination and expression of uncertainty in measurement internationally. This unified approach allows measurements to be compared more easily than beforehand, particularly if there is disagreement of two independently measured results. The NMI Guide provides useful background information on the GUM, as well as pragmatic solutions for measurements in applied environments, without deviating from the GUM philosophy. While there will initially be some work in applying the protocols to the measurements used to test medical equipment, these can easily be programmed into spreadsheets, after which further calculations are straightforward. 138

Gregory et al • A standard approach to uncertainties

Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005

References 1. South Australian Government, Radiation Protection and Control (Ionising Radiation) Regulations 2000, Government Gazette, 2000. 2. Craig, A. R., Heggie, J. C. P., McLean, I. D., Coakley, K. S. and Nicoll, J. J., Recommendations for a mammography quality assurance program, Australas. Phys. Eng. Sci. Med., 24(3):107-131, 2001. 3. McCollough, C. H. and Zink, F. E., Performance evaluation of a multi-slice CT scanner, Med. Phys., 26:2223-2230, 1999. 4. Luketina, I. A., Greig, L., Linear accelerator output variability, Australas. Phys. Eng. Sci. Med., 27(3):155-159, 2004. 5. Nobels, F., Beckers, F., Bailleul, E., De Schrijver, P., Sierens, L., Van Crombrugge, P., Feasibility of a quality assurance programme of bedside blood glucose testing in a hospital setting: 7 years' experience, Diabet. Med., 21(12):1288-91, 2004. 6. International Organization for Standardization, Guide to the expression of uncertainty in measurement, International Organization for Standardization, Geneva, 1995. 7. Hinkle, D., Wiersma, W., and Jurs, S., Applied Statistics for the Behavioural Sciences, Houghton Miffin, Boston, 2002.

8. EURACHEM, Quantifying Uncertainty in Analytical Measurement, Laboratory of the Government Chemist, London, 1995. 9. International Atomic Energy Agency, Quantifying uncertainty in nuclear analytical measurements, TECDOC 1401, IAEA, Vienna, 2004. 10. Bentley R. E., Applying the ISO Guide to the calculation of uncertainty: temperature, National Measurement Laboratory, Sydney, 2001. 11. Taylor, B. N., Kuyatt, C. E., Guidelines for evaluating and expressing the uncertainty of NIST measurement results, National Institute of Standards and Technology, Gaithersburg, 1994. 12. Bentley R. E., Uncertainty in measurement: The ISO Guide, 6th ed., National Measurement Laboratory, Sydney, 2003. 13. Pentz, M., Shott, M., Handling Experimental Data, Open University Press, Milton Keynes, 1988. 14. Kirkup, L., Experimental methods: an introduction to the analysis and presentation of data, John Wiley & Sons, Milton, 1994. 15. International Atomic Energy Agency, Absorbed Dose Determination in External Beam Radiotherapy: An International Code of Practice for Dosimetry based on Standards of Absorbed Dose to Water, TRS-398, IAEA, Vienna, 2000.

139