Ebm

Evidence-Based Medicine Family Medicine Course/IKK3 David E. Newton, MD, MS, DABFP 2005

I. Introduction A. Definition “The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” (Sackett, et al., 1997). B. Role of Evidence-Based Medicine (EBM) 1. Practice of EBM must be integrated with clinical expertise, adding four important elements: a. expertise in performing the patient history and physical exam b. knowledge of the patient, the family, and the community, which provides a context for therapeutic decision-making c. a relationship with the patient informed by his or her beliefs and values d. practical knowledge of the availability of resources in the community 2. EBM provides a framework for knowing when to adopt the new tests and therapies, when to discard old ones, and how to look for answers to the clinical questions that arise every day in the care of patients. a. sources of information don’t always agree: 1) journals 2) textbooks and review articles 3) continuing medical education seminars 4) opinions of colleagues/experts 5) newsletters 6) pharmaceutical representatives 7) practice guidelines 8) personal clinical experience b. the EBM approach assumes that there is a single best answer to each specific patient issue but often one has to be satisfied with the best approximation of the truth. c. EBM approach provides liberation from reliance on dogma and tradition d. emphasizes patient-oriented outcomes over disease oriented evidence 1) when patient-oriented outcomes information is lacking, pathophysiologic reasoning is used.

1

II. Practicing EBM A. Five step process 1. ask a clinical question a. example: “Does patching the eye for a corneal abrasion improve the outcome?” 2. obtain information to answer the question a. determine whether research has been performed to answer the question 3. evaluate the information for relevance and validity a. process for this step is described below 4. apply the information to the patient a. economics, logistics, patient preferences, culture, and research information will all affect the patient management decision. 5. evaluate the effect a. requires analysis of effectiveness of the physician’s practices such as through CQI (Continuous Quality Improvement) 1) implementing CQI in one’s practice requires further training beyond the scope of this lecture material

III. Information Mastery A. Determining the Usefulness of Medical Information 1. Goal is to spend the least amount of time and energy finding a useful and valid answer to our questions. 2. Useful information has three attributes: a. relevant to every day practice b. valid (true) information c. takes little work to obtain 3. Relevance a. POEMs—Patient-Oriented Evidence that Matters come from research that evaluates the effectiveness of interventions on those outcomes that matter the most to our patients b. DOEs—Disease-Oriented Evidence 1) information aimed at increasing understanding of a disease: etiology, prevalence, pathophysiology, pharmacology, prognosis, etc. 2) provides the basis for diagnosing and understanding how to treat/prevent a disease c. Example of DOE vs POEM 1) use of antiarrhythmics for suppression of ventricular arrhythmias a. DOE showed that two new drugs were very effective at suppressing ventricular arrhythmias and were therefore assumed to be effective at reducing the risk of sudden cardiac death. b. Cardiac Arrhythmia Suppression Trial (CAST)

2

1. showed that patients who took these medications were more likely to die than patients who did not take them (Echt et al, 1991). c. available evidence focused on the disease (ventricular arrhythmia) rather than on the patient. Assumptions of benefit to the patient turned out to be false. d. Sometimes DOEs and POEMs lead to similar conclusions but sometimes they lead to opposite conclusions. e. unfortunately, most of the information currently available for treatment of diseases is DOE. f. How to distinguish DOE from POEM 1. see table 14-1 2. determine whether one has to make any assumptions before applying the information. 3. determine whether the information requires thinking or assuming certain outcomes are determinable or by knowing the outcome because clinical trials have demonstrated effectiveness of the intervention on the outcome. 4. one should not assume that treating a disease is always in the best interest of the patient!

3

4. Validity: Studying POEMs a. studying POEMs that evaluate the diagnosis, treatment, or prognosis of an illness seen with some degree of frequency should be top priority. b. the validity of information defines to what extent the knowledge gained is useful. c. well-designed clinical studies that minimize bias are more likely to provide valid conclusions. This is the foundation of the scientific method. d. validity assessment can be done individually or in conjunction with others, or, with great care and caution, it can be delegated to those with the appropriate training and available time. e. potential sources of validity assessment include a colleague, journal club, published rigorous evaluations, such as those in the POEMs section of the Journal of Family Practice, those published by the American College of Physicians (ACP) Journal Club, or those in Evidence-Based Practice.

4

5. Work a. a negative attribute that we must consider when evaluating the usefulness of information is the amount of effort it takes to obtain the information. This includes the cost, or the energy it takes to track down the answer, or the professional pride when asking a specialist a question. g. the goal is to find a source with the highest “usefulness score.” The best source of information provides highly relevant and valid information with minimal effort required to obtain it. Sources such as this are rarely available, so we need to find a balance among the three factors of relevance, validity, and work. Usefulness of Info = (Relevance x Validity) Work to Obtain

B. Understanding Study Results 1. Statistical Analysis a. this has been covered previously in IKK. 2. “P-value” a. used to determine whether the observed difference is “real” or could be due to chance. b. P-value of ≤ 0.05 is a good threshold value for validity 1) means that there is only a 5% chance that the findings do not represent a real effect of the intervention on the outcome studied. 2) the smaller the P-value, the stronger the confidence that the difference seen is real. c. RRR (Relative Risk Reduction) 1) one common method for conveying the degree to which an intervention prevents mortality. 2) this number is the percent reduction in the risk of the studied outcome that was achieved by use of the intervention. 3) these numbers tend to impress because of their size, and therefore the clinical relevance can easily be overstated. d. ARR (Absolute Risk Reduction) 1) represents the absolute difference between two groups being compared 2) often this number is difficult to grasp because it usually involves differences in decimal figures (e.g. 0.088-0.059% or 0.029%). This tends to lead to an underinterpretation of the significance of the effect of the intervention. e. NNT (Number Needed to Treat) 1) conceptually easier to understand and apply

5

2) equals the number of patients who need to be treated/tested, etc to prevent one of them from experiencing the outcome (e.g. death, stroke, myocardial infarction, etc.). 3) easily calculated as 1/ARR 4) for example, a NNT of 3,450 means that 3,450 patients would have to be treated/tested for one patient to benefit from avoiding the outcome measured. 5) The NNT is an easy way to grasp the magnitude of an effect, especially when it can be compared to the NNT for other interventions. 6) The lower the NNT, the more significant the effect, which means that fewer patients would have to be treated/tested to benefit one of them. 7) when the incidence of the outcome is low, the NNT is likely to be high and therefore, the benefit of the treatment is smaller (of course, the opposite is also true). 8) See Table 14-2 for examples of the NNTs for various medical interventions.

C. Sources of Information Different sources of information will be more useful than others, based on a balance of relevance, validity, and work. Each category can be useful in certain circumstances.

1. Journals a. information tends to be highly valid b. relevance may not be high c. work to read journals tends to reduce usefulness, depending on the type of journal 1) Knowledge Creation Journals a. publish studies that present new data and information or reviews that carefully synthesize knowledge from previously published research. b. tend to be highly valid but relevance is often not as high and work is high.

6

c. Examples: the New England Journal of Medicine, JAMA, the Journal of Family Practice, etc. (at least 4000 are listed in the US National Library of Medicine) d. useful when looking for an answer to a specific question. e. research studies in knowledge creation journals can be quickly skimmed for relevance by reading the title of the article and the abstract. f. the initial screen of the article should focus on the answers to three important questions: 1. “Did the authors study an outcome that my patients should care about?” 2. “Is the problem studied one that is common to my practice, and is the intervention feasible?” 3. “Will this information, if true, require me to change my current practice?” g. This simple screen can eliminate most of the articles in knowledge creation journals (Ebell et al., 1999) 2) Translation Journals a. consist mainly of expert reviews of current knowledge or more readable summaries of research findings published in other journals. b. the major advantage is that the work to read them is lower while the relevance tends to be higher. The validity can suffer, depending on the quality of the job done by the reviewer. c. translation journals enjoy massive and preferential readership in all medical specialties (Radulescu, 1989). d. For the busy physician, they offer short articles that can be quickly read and understood. e. Information that might require a change in practice should be cross-checked with other sources. f. be observant for phrases that hedge on the validity of a recommendation: “It seems,” “may be effective,” “should be useful,” “so one must assume,” “it appears,” “it may be,” etc., signal that the evidence is not as strong as it should be.

2. Review Articles a. excellent method of surveying the medical literature on a topic which dramatically lowers the work of covering a large amount of information.

7

b. Textbooks can be good collections of review articles. 1) textbooks tend to lose validity quickly because they take so long to be published after the original research is conducted. 2) they are useful for reviewing previously learned material or for finding information on a topic that has not changed recently. c. Summary Reviews broadly paint the landscape of the topic under discussion, whereas Synthesis Reviews focus on one question and attempt to fashion an answer. d. summary reviews can have validity problems. 1) Many authors who write summary reviews have already determined a conclusion, and have merely searched the literature for research to support that contention. 2)The likelihood of bias in summary reviews is high. e. Synthesis Reviews, on the other hand, are usually either “metaanalysis” or a “systematic review.” 1) authors generally review all the research relevant to the topic. Each individual study results are treated as though they were a part of one large trial. 2) this approach can yield results that could not be obtained by reviewing individual trials only. 3) For example, 19 of 23 trials on the use of beta blockers after myocardial infarction did not show a benefit. However, when the results of all of the studies were combined, beta blocker therapy was associated with a 23% reduction in risk of death (Yusuf et al., 1985). f. The Cochrane Library is one of the best sources of high-quality reviews. 1) the Cochrane Collaboration is a mixture of volunteer and supported efforts from around the world. Its aim is to provide a clearinghouse for the best, most clinically relevant research information. 2) The Cochrane Database of Systematic Reviews is the most useful feature of the Cochrane Library. a) each review focuses on answering one specific question b) only results of randomized, controlled trials are used in the reviews.

3. Continuing Medical Education Seminars a. Although many people leave a CME presentation feeling that they have learned something, research has not shown that learning actually occurs and practice habits are infrequently influenced by CME (Davis et al. 1992) b. another problem is validity

8

1) It is frequently difficult to separate the speaker’s personal anecdotal information from clinically supported evidence. Attendees should take an assertive approach by asking questions to uncover the level of evidence. 2) Attendees should look for conclusions/recommendations based on POEMs not on the speaker’s clinical experience. 3) care must be taken not to rely on the expertise of the speaker as the sole support of the credibility of the evidence.

4. Experts a. some experts have a wealth of experience and knowledge in a particular area and can be called content experts. b. information from content experts tends to be very subjective c. experts frequently do not agree with one another d. the most helpful type of expert could be called a YODA (Your Own Data Appraiser) 1) these are wise physicians who are able to combine their knowledge, gained by experience, with a careful and complete understanding of available research. 2) an expert who can offer this combination of wisdom and accompanying evaluation of evidence should be highly prized.

5. Newsletters a. provide abstracts and sometimes commentary on articles of interest to family physicians. b. as with any source of information, one must evaluate the relevance and the validity of the material. c. examples include: Evidence-Based Practice, The ACP Journal Club, Journal Watch, Drug Therapy Update, The Medical Letter, Medical Sciences Bulletin.

6. Pharmaceutical Representatives a. a helpful mnemonic for gathering information about a new drug therapy from a pharmaceutical representative is “STEPS,” which will help evaluate the usefulness of a new drug: 1) Safety 2) Tolerability 3) Effectiveness 4) Price 5) Simplicity b. although PR’s tend to give biased information if allowed to control the presentation, an assertive approach on the part of the physician can sometimes yield useful information from the literature. PR’s often are willing to find useful articles requested by the physician that pertain to a new drug.

9

7. Practice Guidelines a. tend to be systematically-developed guides to medical practice derived from textbooks, review articles, clinical experience, etc. b. they are most effective when used by the physicians who help develop them. c. the best use is when a guideline is used as suggestions governing most practice, most of the time and are not thought of as inflexible protocols.

8. Clinical Experience a. before using evidence gained from clincal experience to justify an approach to every patient, we must apply the rigors of the scientific method. There are numerous examples in the history of medicine in which practices that seemed to be beneficial were found, with research, to be either ineffective or harmful. b. the opposite error can also occur: when a physician assumes that a practice that is useful in general is required fro each individual patient. Very few interventions in medicine are helpful in 100% of patients. Individual patients may not match up with patients used in a research trial. c. clinical science and clinical experience are inseparable. Neither rigid adherence to guidelines nor practice patterns developed solely from experience are likely to result in the best patient outcomes. Clinical experience can augment but not replace knowledge obtained through careful study.

9. Web-Based Sources of Evidence-Based Clinical Information

10

IV. Using Information to Change Practice Patterns A. Importance of Changing Practice Patterns 1. What might have been correct 5, 10, 15, or 20 years ago was based on the best information available at that time. As new information becomes available, approaches to patients have to change as well. 2. The Science of Medicine involves identifying the best information that comes to bear on the care of patients. The best information is that which comes from the best scientific method in medicine—the randomized, controlled trial. 3. The Art of Medicine is the decision-making ability necessary to apply scientific information to the care of patients, taking into account their specific needs and desires. B. Barriers to Change 1. fear, discomfort with ambiguity, guilt, loss of control are the top reasons for resistance to change (Argyris, 1991, Conner, 1993). 2. lack of awareness of information. 3. fear of unintended consequences 4. stubbornness. C. Managing Change 1. See Table 14-5

11

2. A high state of evidence is not present for most disorders, leaving room for a combination of patient preference, personal experience, inductive reasoning. In these instances, it is appropriate for each physician to tailor care based on their own assessments.

V. Conclusion Using POEMs as our guide to which services to offer and which to leave out can clearly help eliminate the waste in medicine, and, in so doing, help improve the fairness in distribution of resources. Without evidence that many interventions result in improved patient outcomes, we are raising the cost of health care without the confidence of any benefit to our patients, and, in some cases, harming them. The beauty of basing clinical practice on POEMs, is that we can improve the quality of health care provided to our patients while, at the same time, reducing costs by eliminating wasteful, unnecessary or harmful interventions. The result is better outcomes for our patients and society in general.

12