How To Really Measure Cost-effectiveness

HOW TO REALLY MEASURE COST-EFFECTIVENESS

THE VALUE QUESTION • Is a medical technology good value for the cost? – Are buying sufficient health for what we are spending?

Henry Glick, Ph.D. University of Pennsylvania 32 Annual Symposium “Clinical Microbiology Update 2001”

* Comparison of costs and effects with other therapies for the same disease * Comparison with other therapies for different diseases

• Need to know the value of the outcome we are assessing – Is spending $5,000 to increase the accuracy of a test by 1 percentage point good value or not?

COMMON METHODS

IMPLICATIONS • The need to make comparisons with other diseases and to know the value of the outcome implies final outcomes more important than intermediate ones – Years of life saved – Quality-adjusted years of life saved (QALYS)

• Other outcomes (e.g., cost per correct diagnosis) can be calculated, but will be of less use when arguing for the value of medical technologies

• Decision analysis e.g., Frazier et al. Cost-effectiveness of screening for colorectal cancer in the general population. JAMA 2000;284:1954-61.

• Randomized trials • References: Economic Evaluation in Health Care: Merging Theory with Practice. Ed. M Drummond and A McGuire. Oxford University Press, 2001. Cost-Effectiveness in Health and Medicine. Ed. MR Gold, et al. Oxford University Press, 1996.

TYPES OF MODELS

DECISION ANALYSIS • Formal technique for combining data from several sources using simulation models • Abstract relevant parts of reality • Forces the identification of options and potential outcomes

• Decision trees – Graphic representation of a set of

equations •

Survival function models

•

State transition or Markov models

• Forces decision maker to make assumptions explicit

1

Treat Vs. Test and Treat Disease + Test and

0.2

Treat 0.8 Disease + Treat Empirically

Test -

0.1 Disease

Treatment Decision

Test +

0.9

Test +

0.15 Test -

0.85

$26,000 5 QALYS $10,000 1 QALYS $16,000 14 QALYS $1,000 15 QALYS $25,000 5 QALYS

0.2

Disease -

$15,000 14 QALYS

0.8

Treat Vs. Test and Treat Disease + Test and Treat

0.2

Test +

0.9 Test -

0.1 Disease Test + 0.15 0.8

Treatment Decision

Disease + Treat Empirically

0.2

Disease 0.8

Test -

0.85

$26,000 5 QALYS $10,000

STEPS IN THE DEVELOPING A DECISION TREE 1. Develop the structure of the model (e.g., draw the decision tree) 2. Identify probabilities that different events in the model occur 3. Identify the outcome values 4. Analyze the tree (evaluate the principal analysis / analyses) 5. Perform sensitivity analysis

SOURCES OF PROBABILITIES • Published reports of clinical and epidemiological studies – Intervention Study

1 QALYS

* Natural history of disease

$16,000 14 QALYS

* Effect of intervention

$1,000 15 QALYS $25,000 5 QALYS

$15,000 14 QALYS

IDENTIFYING THE OUTCOMES

– Intervention Study and Observational Study * Natural history: observational study * Intermediate effect of intervention: intervention study * Final effect of intervention: observational or intervention study

• Consensus of expert opinion • Personal opinion

RESULTS

• Costs – Direct medical costs – Direct nonmedical costs – Indirect costs

• For each strategy: – Expected costs – Expected outcomes

• Effects – Survival – Years of survival – Quality-adjusted years of survival – Intermediate measures such as sensitivity/specificity/ likelihood ratios

• Comparison of costs and effects – Difference in expected costs – Difference in expected effects – Ratio of difference in expected costs and effects

2

SENSITIVITY ANALYSIS

POTENTIAL LIMITATIONS

• Confidence intervals – “Second-order” Monte-Carlo simulation

• When events can repeat themselves (e.g., when one can develop cancer this year, next year, etc.), decision trees may become too “bushy”

• Robustness of results • Threshold analysis

• Trees generally don’t directly account for timing of events (e.g., that one can develop cancer this year, next year, etc.) • Markov or state transition models may help overcome these problems

MARKOV OR STATE TRANSITION MODELS

Colorectal Cancer States Polyps

• Describe the disease process by simplifying it into a set of states • Disease progression represented probabalistically as a set of transitions among the states during fixed intervals of time (e.g., 6 month or 1 year intervals) • Effects of an intervention (e.g., a diagnostic test to detect polyps and cancer) described as a change in the transition probabilities

Local

Death

CA

Regional

Distant

CA

CA

• Assess resource use and QALYS as a function of being in a state for a period

STEPS IN DEVELOPING A MARKOV MODEL 1. Enumerate the states 2. Define allowable state transitions 3. Associate probabilities with the transitions 4. Identify a cycle length 5. Identify an initial distribution of patients within the states 6. Identify the outcome values 7. Analyze the Markov model 8. Perform sensitivity analysis

No Disease

RANDOMIZED TRIALS • Considered the gold standard for evaluating the efficacy, effectiveness, and efficiency of medical technologies • Potential advantages: – Provide head-to-head comparison between technology under evaluation and alternative – Collect costs and outcomes in same individuals, and maintain correlations between costs and outcomes

• Potential disadvantages – May reflect idealized rather than usual practice – May not provide sufficient follow-up to determine final outcomes

3

STEPS IN CONDUCTING ECONOMIC EVALUATION IN RANDOMIZED TRIAL • Step 1: Quantify the costs of care

FOUR ISSUES • What health care resource use should be measured (data collection)?

• Step 2: Quantify outcomes

• How should the resources be valued? • Step 3: Assess whether and by how much average costs and outcomes differ among the treatment groups • Step 4: Compare magnitude of differences in costs and outcomes and evaluate "value for the cost" (e.g., by reporting a cost effectiveness ratio or the probability that the ratio is acceptable)

• How many subjects should be included in the study (sample size)? • For time-limited trials of therapies with longerterm effects: How should long-term results be projected?

• Step 5: Conduct sensitivity analysis

WHAT HEALTH CARE RESOURCE USE SHOULD BE MEASURED? • Ideal: All resource use • In practice: – Services that make up a large portion of the difference in treatment between patients randomized to the different therapies under study * Estimate of the cost impact of the therapy

– Services that make up a large portion of the total * Reduces likelihood that differences among unmeasured services will lead to bias * Provides measure of variability of costs

HOW MANY SUBJECTS SHOULD BE INCLUDED IN THE STUDY? • Early 1990's approach: Select the larger of the sample sizes needed for estimating pre-specified cost and effect differences – i.e., what sample size was required to identify a $1000 difference in costs, and what was required to identify a 10% reduction in mortality

• Current approach: Estimate the number of study subjects needed to rule out unacceptably high upper confidence limits for the cost-effectiveness ratio

HOW SHOULD THE RESOURCES BE VALUED? • SOURCES OF U.S. UNIT COST DATA – Hospital charges adjusted using cost to charge ratios – Internal hospital costing system data – Diagnosis related group payments (Hospital stays) – Resource-based relative value units (Physician services/procedures) – Trial-specific costing exercise (e.g., time and motion studies

SAMPLE SIZE DATA REQUIREMENTS • Difference in costs (old and new) • Difference in effects (old and new) • Standard deviation of costs (old and new) • Standard deviation of effects (old and new) • Correlation of costs and effects (new)

4

HOW SHOULD LONG-TERM RESULTS BE PROJECTED?

$6,277

15000

$22,971

Incremental Costs

12000 9000

$4,407

6000

– Maintain the same time horizons for costs and outcomes observed in the trial

$1,140

3000 0 -3000 -0.10

0.40

Point Estimate CER: $5700

0.90

1.40

1.90

• For time-limited trials of therapies with potentially long term effects, one should evaluate the costs and outcomes that were observed during the trial

2.40

Incremental QALYS

Means and S.D.s for identical

• Because the relative magnitude of incremental costs and outcomes observed during the trial may not be reflective of the relative magnitude for longer periods of follow-up, one should also project the results for longer periods

Correlations: 0.988; -0.988

COST EFFECTIVENESS RATIO AND 95% CI, 4S FOR DIFFERENT LENGTHS OF FOLLOW-UP/PROJECTION _____________________________________________________ Years of Follow-up Point Estimate 95% CI _____________________________________________________ 2

$282,857

$45,577 to Dominated

4

$12,074

Dominates to Dominated

5.5

$15,258

Dominates to $122,772

10

$12,246

Dominates to $42,263

20 $7,320 $681 to $21,841 _____________________________________________________

CONCLUSIONS • Economic evaluations attempting to assess value for the cost • Given need to compare interventions for a single disease and among diseases, a common outcome such as survival or quality adjusted survival should be used • Common techniques for evaluation include decision analysis and randomized trials • Lots of opportunities exist for evaluating microbiologic technologies

5