Interpreting Data Mho March 09

Learning outcomes  Recap data collection and analysis  Appreciate psychometric properties    

and their calculation Identify and interpret descriptive statistics Decipher basic inferential statistics Explain some research jargon Apply new knowledge to the evidence

Session plan

Task 1  Write down up to 5 issues that you

think are important when interpreting data  In a group of 4 discuss which you

think are most important and why?

Study aim  What is the research question looking

for?  Difference?  Correlation?

Linear association  Prediction?

Regression  Agreement?

Kappa ICC

Nominal  The lowest and simplest level of    

measurement Used to classify or label people, creatures, behaviour or events Uses categories that are mutually exclusive e.g. male/female; dead/alive Sufficient categories needed to allow every observation to be assigned Often includes ‘other’ category

Ordinal  Indicates a rank order in which things are arranged.  from the greatest to the least  the best to the worst

 Allows presentation of the order of the observations

but does not provide information on actual values  For example, Motor assessment scale, Barthel index, FIM & FAM

Interval  This is a true unit of measure  Conveys the order of the observations

AND indicates the distance or degree of difference between the observations  Does not have an absolute zero  The difference between each score is equal for example, temperature (degrees centigrade)

Ratio  Provides the most precise information of

all and includes the maximum amount of information, again a true unit of measure  Has an absolute zero point that has real meaning, therefore offers an absolute measure  The zero point dictates the absence of the property measured  eg. height, weight, speed

Type of data

P ro p e rty C a te g o rie s m u tu C a te g o rie s lo g ic

Adapted from Puri 2002

Types of data  What type of data are the following?  Age  Course of study  Social Class  Year  Weight  Height  Profession  Adult shoe size  Pain / functional disability measure

Data analysis  The process of gathering,

modelling, and transforming data with the goal of highlighting useful information, suggesting conclusions, and supporting decision making

Data analysis  Descriptive statistics  Used to describe the data in a sample,

e.g. mean, median, standard deviation.  Refer to any statistics textbook to gain an understanding of appropriate use

 Inferential statistics  Infer findings from the sample to the

population

Descriptive statistics  Line  Bar  Histogram

 Pie chart  Scattergram  Box plot

Scattergrams  Two-dimensional representations of the

relationship between pairs of variables,  The graph represents the points at which the two variables intersect for each case in the sample.  Easy visual representation of 3 aspects of a pairwise relationship:    

Whether or not it is linear Whether it is positive or negative The strength of the association. They can be useful aids to the understanding of the idea of correlation

Scattergram example 90

80

70

age at onset

60

50

40 0

5

H.A.D.S Anxiety

10

15

20

25

Boxplots  Also known as a box-and-whisker

diagram it is a convenient way of graphically depicting groups of numerical data.  Can be useful to display differences between populations  The spacings between the different parts of the box help indicate the degree of dispersion  Displays five summaries of the data

Boxplot example 100 90 80 70

age at onset

60 50 3

24

40 30 N=

21

5

Missing

6

8

1 0

8

5

3 2

H.A.D.S. Anxiety

4

9

5 4

5

6

7 6

10

2

9 8

3

4

11 10

1

2

13 12

1

1

15 14

1

20 17

Inferential statistics  Inferential statistics or statistical

induction comprises the use of statistics to make inferences concerning some unknown aspect of a population  Data can be categorised

Inferential statistics  Decisions which need to be made:  Qualitative or quantitative?  Difference or Correlation?  Type of data:  Parametric – continuous data  Non-parametric – continuous data  Ordinal  Categorical  Number of groups in the sample  Paired/ Un-paired data

Qualitative or quantitative?  Quantitative:  Data may be represented numerically

 Qualitative:  Numerical representation is insufficient  Require words or even images  Examples include personal experiences,

life story, perceptions

Difference or correlation?  Difference  Self-explanatory!  Example: Is early mobilisation more

effective than deep breathing exercises post operatively

 Correlation  Looking for a relationship between variables  Example: smoking cessation and

improvement in respiratory function

Parametric data  Conditions:  Interval or Ratio Data  Normally distributed:

1250

1000

data is normally distributed:

750

Count

 Various ways to test if

 Mean/ 2 S.D's Kolmogorov-Smirnov Shapiro-Wilk

500

250

66.00

67.00

68.00

69.00

Height (ins)

70.00

71.00

Paired/ un-paired data (Repeated measures)/ (Independent samples)

 Paired data are often the result of

before and after situations - same measurement on the same person on 2 different occasions.  Perceived stress level of students on

different programs of study.  Measurements of muscle strength before and after an exercise to fatigue the muscle.  Attitudes of males and females to physiotherapy.

Start

Type of data

1 group Groups of numerical data

One sample t-test number of groups 2 groups

Categorical data

More than 2 groups

Chi-Square test.

Paired data? Yes Non Parametric parametric Repeated Friedman Measures ANOVA

N o Parametric

Paired data? Non parametric

Yes

Look for differences No Non Parametric parametric

One-Way Kruskal-Wallis ANOVA

t-test Mann (unrelated)Whitney Correlations Differences or Correlations

Differences

Parametric

Non parametric

Parametric

Non parametric

Pearson

Spearman

t-test (related)

Wilcoxon

p-value  Output of an inferential statistical test  p (probability) value is used to assess how

likely the results we have obtained are due to chance  Conventionally set at 0.05, or 5% chance that results obtained from sample are due to chance  This is arbitrary and open to criticism  However, important concept to be aware of

Confidence intervals  This is how confident we are our sample     

represents the population 95% CI can be calculated for given data from our sample. Usually presented in parenthesis eg CI = (O.8,2.7) So this would mean that 95% of the time the mean will be between 0.8 and 2.7 A narrow CI implies greater precision This result would be non-significant as the CI does not cross 0.

Sample size calculation  Identifies the sample size required for study  Smaller samples show greater variance  Calculated from the primary outcome

measure and previous evidence of its SD in the population being investigated  Takes into account the relative statistical significance and the power of the study  Often the reason for a pilot RCT  Ethically important

Blinding  Single  the researcher knows the details of the

treatment but the patient does not

 Double  one researcher allocates a series of numbers to

'new treatment' or 'old treatment'. The second researcher is told the numbers, but not what they have been allocated to.

Randomization  Involves the random allocation of

different interventions (treatments or conditions) to subjects.  As long as numbers of subjects are sufficient, this ensures that both known and unknown confounding factors are evenly distributed between treatment groups.

Psychometric properties  The elements that contribute to the

statistical adequacy of the study in terms of  Reliability  Validity  Internal consistency  Responsive to change

Reliability  Data is reliable if it has been shown to be

reproducible with the same/similar results  Reliability is inversely proportional to random error  Types of reliability  A measure gives the same results on repeated tests by

an individual ( if the respondent has not changed)  A measure gives the same result if different individuals apply it ( at the same time)

Inter rater reliability  Inter rater reliability is assessed by the

degree of agreement between the 2 sets of scores

 Often assessed using Pearson's or Intra

Class Correlation

 Indicates the strength and direction of a linear

relationship between two random variables.  However, this correlation assesses association between 2 measurers rather than agreement.  For Continuous data

Inter rater reliability  Can also be measured using Cohen’s

Kappa coefficient

 Kappa measures the percentage of data

values in the main diagonal of the table and then adjusts these values for the amount of agreement that could be expected due to chance alone.  For categorical data  Weighted Kappa for ordinal data

Interpreting kappa  Kappa is always less than or equal to 1. □

A value of 1 implies perfect agreement and values less than 1 imply less than perfect agreement.

 Kappa can be negative. This is a sign that the

two observers agreed less than would be expected just by chance.  It is rare that we get perfect agreement.  Different people have different interpretations as to what is a good level of agreement.

Responsiveness  Considers the ability to detect

change (that is meaningful to patient)  Simplest way to test is to correlate change scores from the measure with changes in other available measures but is this responsiveness or just the ability to show change

Validity  The degree to which a test measures

what it was designed to measure.  The degree to which a study supports the intended conclusion drawn from the results  Types of validity  internal  external

 May be recorded as convergent and

discriminant validation

Validity  Many measures have multiple scales

within them considering different constructs  Ensuring the internal structure of the measure is also construct validity and is measured through factor analysis.  This looks at the patterns of items within a measure that together assess a single underlying construct

Internal consistency  A measure usually has several items  Based on the principle that several

observations are more reliable than one  The items need to be homogeneous  One approach – split items randomly into 2 halves and assess agreement  Cronbach’s Alpha Coefficient estimates the average agreement between all possible ways of splitting the 2 halves.

Summary  Identify study aim  What are they looking for

 Check type of data collected  Nominal, interval etc  Parametric, non-parametric

 Are they using the appropriate test  Consider influencing factors  Psychometric factors  Sample size, Blinding, Randomization

Task 2  In groups of four  Design a study

 Consider  What you want to investigate  What you are measuring  What type of data you are collecting  What test would be appropriate in

assessing the psychometric properties of your outcome measure

Intention to treat (ITT) analysis

 An analysis based on the initial

treatment intent, not on the treatment eventually administered.  ITT analysis is intended to avoid various misleading artifacts.  For example, if people who have a more

serious problem tend to drop out at a higher rate, even a completely ineffective treatment may appear to be providing benefits if one merely compares those who finish the treatment with those who were never enrolled in it.

.

Intention to treat (ITT) analysis

 For the purposes of ITT analysis,

everyone who begins the treatment is considered to be part of the trial, whether they finish it or not.  Full application of intention to treat can only be performed where there is complete outcome data for all randomized subjects.  Although intention to treat is widely cited in published trials, it is often incorrectly described and its application may be flawed.

Summary  Recapped principles of study design,

data collection and statistical analysis  Considered influencing factors  Applied knowledge to devise a study into the dunkability of biscuits  Reviewed how data may be presented