1.12 Protocolo Colaborativo Michigan

Feature Articles

Protocol-Based Resuscitation Bundle to Improve Outcomes in Septic Shock Patients: Evaluation of the Michigan Health and Hospital Association Keystone Sepsis Collaborative* Michael P. Thompson, PhD1,2; Mathew J. Reeves, PhD1; Brittany L. Bogan, MHSA, CPPS2; Bruno DiGiovine, MD, MPH3; Patricia J. Posa, RN, BSN, MSA4; Sam R. Watson, MSA, CPPS2 Objectives: To evaluate the impact of a multi-ICU quality improvement collaborative implementing a protocol-based resuscitation bundle to treat septic shock patients. Design: A difference-in-differences analysis compared patient outcomes in hospitals participating in the Michigan Health & Hospital Association Keystone Sepsis collaborative (n = 37) with noncollaborative hospitals (n = 50) pre- (2010–2011) and postimplementation (2012–2013). Collaborative hospitals were also stratified as high (n = 19) and low (n = 18) adherence based on their overall bundle adherence. *See also p. 2275. 1 Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI. 2 Keystone Center for Patient Safety & Quality, Michigan Health & Hospital Association, Okemos, MI. 3 Division of Pulmonary, Critical Care, and Sleep Medicine, Henry Ford Health System, Detroit, MI. 4 St. Joseph Mercy Hospital, Ann Arbor, MI. The Michigan Health & Hospital Association Keystone Center receives unrestricted donations from Blue Cross Blue Shield Foundation of Michigan. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal). Dr. Thompson disclosed other support and received support for article research from Blue Cross Blue Shield Foundation of Michigan (The Michigan Health & Hospital Association [MHA] Keystone Center receives unrestricted donations). His institution received funding from the Blue Cross Blue Shield Foundation of Michigan (The MHA Keystone Center receives unrestricted donations). Dr. DiGiovine received funding from the Michigan Hospital Association, American Board of Internal Medicine, and Law firm of Silver Golub & Teitell. He has a family disclosure (his wife owns stock in United Medical Systems). Dr. Posa received other support (Surviving Sepsis Campaign Society of Critical Care Medicine-ICU Liberation Collaborative Sage Products Excelsior Medical) and received funding from Michigan Hospital Association, Johns Hopkins Armstrong Institute, and Missouri Patient Safety Organization. She is a Consultant for Advanced Nursing LLC. Dr. Watson’s institution received funding from Cross Blue Shield of Michigan. The remaining authors have disclosed that they do not have any potential conflicts of interest. For information regarding this article, E-mail: [email protected] Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved. DOI: 10.1097/CCM.0000000000001867

Critical Care Medicine

Setting: Eighty-seven Michigan hospitals with ICUs. Patients: We compared 22,319 septic shock patients in collaborative hospitals compared to 26,055 patients in noncollaborative hospitals using the Michigan Inpatient Database. Interventions: Multidisciplinary ICU teams received informational toolkits, standardized screening tools, and continuous quality improvement, aided by cultural improvement. Measurements and Main Results: In-hospital mortality and hospital length of stay significantly improved between pre- and postimplementation periods for both collaborative and noncollaborative hospitals. Comparing collaborative and noncollaborative hospitals, we found no additional reductions in mortality (odds ratio, 0.94; 95% CI, 0.87–1.01; p = 0.106) or length of stay (–0.3 d; 95% CI, –0.7 to 0.1 d; p = 0.174). Compared to noncollaborative hospitals, high adherence hospitals had significant reductions in mortality (odds ratio, 0.84; 95% CI, 0.79–0.93; p < 0.001) and length of stay (–0.7 d; 95% CI, –1.1 to –0.2; p < 0.001), whereas low adherence hospitals did not (odds ratio, 1.07; 95% CI, 0.97–1.19; p = 0.197; 0.2 d; 95% CI, –0.3 to 0.8; p = 0.367). Conclusions: Participation in the Keystone Sepsis collaborative was unable to improve patient outcomes beyond concurrent trends. High bundle adherence hospitals had significantly greater improvements in outcomes, but further work is needed to understand these findings. (Crit Care Med 2016; 44:2123–2130) Key Words: evaluation; outcomes research; quality improvement; resuscitation bundle; septic shock

S

evere sepsis and septic shock are common and highly fatal conditions, which place substantial burden on the healthcare system (1–3). Great strides in the identification and treatment of septic shock have been made in the years since the formative 2001 study by Rivers et al (4), which developed the early goal-directed therapy protocol to treat septic shock patients. Protocol-based therapy—especially time-dependent treatment bundles—remains the mainstay of www.ccmjournal.org

2123

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Thompson et al

clinical guidelines for the treatment of septic shock patients issued by the Surviving Sepsis Campaign (SSC) (5). A recent SSC publication stated that increased compliance to a protocol-based therapy bundle was associated with a 25% relative risk reduction in in-hospital mortality (6). Despite the widespread promotion of protocol-based therapy as the standard of care, there is substantial controversy surrounding its efficacy. Recent randomized clinical trials have shown no benefit of protocol-based therapies on mortality or length of stay compared to usual care (7–9). There is also concern that the protocol-based therapies may lead to inappropriate antibiotic use, unnecessary testing, and overuse of invasive treatments (10). Additionally, successful implementation of protocol-based therapies is particularly difficult (11–17). Common barriers to implementation include difficulty in identifying septic shock patients, lack of knowledge about septic shock in hospital staff, limited resources, and the complexity of the protocol itself (18–20). In 2010, the Michigan Health & Hospital Association (MHA) Keystone Center developed a multi-ICU quality improvement (QI) collaborative project called the “MHA Keystone Sepsis Collaborative.” This program promoted the use of a protocolbased resuscitation bundle in the ICU. To assess the impact of the collaborative, we conducted a difference-in-differences (DID) analysis to compare relative changes in outcomes (mortality and length of stay) in collaborative hospitals with changes observed in noncollaborative hospitals. Furthermore, to explore how the level of resuscitation bundle adherence in hospitals influenced changes in outcomes, we stratified collaborative hospitals into high and low adherence hospitals based on their hospital-level adherence.

METHODS MHA Keystone Sepsis Collaborative In 2010, all Michigan hospitals with ICUs were invited to participate in the MHA Keystone Sepsis collaborative. Once enrolled, ICUs formed multidisciplinary teams, which included physicians, nurses, pharmacists, respiratory therapists, and other staff involved in patient care, such as clerks and technicians. Teams were provided with an informational toolkit on the clinical presentation of sepsis, evidence-based treatment of sepsis, and conceptual framework for the resuscitation bundles. Clinical screening and data abstraction tools were provided to standardize septic shock case definitions and data collection methods between hospitals. Face-to-face workshops followed by regular coaching calls led by clinical experts educated sepsis teams on the informational toolkit information and promoted continuous QI. To foster implementation of the intervention, MHA Keystone employed two previously successful QI strategies: the four E’s in translating evidence into practice (Engage, Educate, Execute, and Evaluate) (21) and a Comprehensive Unit-Based Safety Program (CUSP) (22). CUSP engages and empowers staff to improve patient safety and has previously been shown to improve organizational culture and patient safety and outcomes in ICUs (23). 2124

www.ccmjournal.org

Collaborative hospitals collected patient-level resuscitation bundle compliance data on patients admitted to the ICU with a diagnosis of septic shock, which were submitted monthly through a secure online portal maintained by the MHA. The eight bundle measures reflected the 2008 SSC guidelines: fluid bolus administration, lactate measurement, antibiotic administration, obtaining two blood cultures, blood cultures obtained prior to antibiotic administration, and clinical achievement in central venous pressure (> 8 mm Hg), mean arterial pressure (> 65 mm Hg), and central venous oxygen saturation (> 70%) (24). No additional demographic or clinical information were collected. Baseline data collection began in April of 2011, and full uptake of the intervention was expected by the end of 2011. The MHA Keystone Sepsis collaborative concluded at the end of 2013, but a second iteration is ongoing, and is part of the Blue Cross Blue Shield of Michigan pay-for-performance program. Data Source and Sample Discharge data from the Michigan Inpatient Database (MIDB) provided outcome and covariate data for hospitals participating in the Keystone Sepsis collaborative as well as noncollaborative hospitals. The MIDB is a comprehensive source of all-payer, patient-level data on all acute-care hospital discharges in the State of Michigan (25). Between 2010 and 2013, we identified 49,232 patients from 87 hospitals with ICUs who had the International Classification of Diseases, 9th Edition code for septic shock (785.52) listed within the first 10 diagnosis codes (1). Patients were excluded from analysis if they were under the age of 16 (n = 724; 1.5%), left against medical advice (n = 185; 0.4%), had a hospital length of stay of more than 90 days (n = 205; 0.4%), or had no information on discharge disposition (n = 8; 0.02%). After exclusions, n equal to 48,110 septic shock cases remained for analysis. Since this project utilized existing data with no identifiable protected health information, it was deemed exempt from institutional review board review. Outcomes of Interest The primary outcomes of interest in this analysis were in-hospital mortality and hospital length of stay (d), obtained from the MIDB. Independent Variables For this analysis, we defined two independent variables: preversus postimplementation period and hospital participation in the collaborative. Since full implementation of the collaborative was expected by the end of 2011, if patients were discharged in either 2010 or 2011, they were considered preimplementation patients, and postimplementation if discharged in 2012 or 2013. Hospitals were defined as collaborative hospitals if they were enrolled in the Keystone Sepsis collaborative (n = 37). Using bundle adherence data submitted to the MHA portal, we calculated the average resuscitation bundle adherence for each collaborative hospital as the sum of all bundle measures each patient received divided by the sum of all December 2016 • Volume 44 • Number 12

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Feature Articles

measures for which each patient was eligible. To explore how the level of resuscitation bundle adherence influenced changes in outcomes, we stratified collaborative participating hospitals as either high or low adherence based on a median split of hospital-level average bundle adherence. Statistical Analysis We compared demographic, clinical, and hospital characteristics from the MIDB for patients treated in collaborative (as well as high and low adherence hospitals) compared to noncollaborative hospitals in the pre- and postimplementation periods of the collaborative using chi-square tests and analysis of variance for categorical and continuous variables, respectively. Patient demographics included age, gender, and insurance payer (Medicare, Medicaid, private, other). Clinical information included admission type (emergency, urgent, elective, unknown), transferred patient status, and Charlson comorbidity index (26, 27). Hospital characteristics included rural versus urban location, teaching status, bed size (< 100, 100–300, and 300+), proportion of Medicaid patients, bed size, and teaching status. We estimated unadjusted pre- versus postimplementation effects in collaborative and noncollaborative hospitals, as well as high and low adherence hospitals using logistic regression for in-hospital mortality, and linear regression for hospital length of stay. We then adjusted pre- versus postimplementation effects for patient characteristics, and then patient and hospital characteristics. Risk-adjusted in-hospital mortality rates and hospital length of stay pre- and postimplementation were estimated using model-based indirect standardization. To evaluate the impact of participation in the collaborative on in-hospital mortality and hospital length of stay, we performed a DID analysis. The DID method is a well-cited and highly robust method that compares relative changes in outcomes observed in the intervention group (difference 1) with concurrent changes observed in a control group undergoing similar temporal trends but are not exposed to the intervention (difference 2) (28, 29). Conceptually, the independent effect of the intervention is referred to as “the DID estimator,” which represents the “DID” between intervention and nonintervention groups pre- and postimplementation (difference 1 – difference 2 = DID estimator). In practice, the DID estimator is derived from regression models and is represented by an interaction term between an intervention indicator (i.e., hospital collaborative status) and indicator for pre- versus postimplementation period. Using regression models also allow for adjustment on patient and hospital characteristics. A statistically significant, nonzero interaction term would indicate that relative changes in outcomes observed in the intervention group were significantly different than relative changes observed in the control group. Further explanation of the DID analysis can be seen in the Supplementary Appendix (Supplemental Digital Content 1, http://links.lww.com/CCM/B930). The DID analysis was performed for both outcomes comparing collaborative versus noncollaborative hospitals, as well as high and low adherence versus noncollaborative hospitals. Critical Care Medicine

In a sensitivity analysis, we reran the DID analysis for length of stay excluding patients who died, as length of stay may be shorter for patients who died in the hospital. We also excluded 2011 data from the preimplementation period to eliminate any spillover of the intervention into the preimplementation period. To test the parallel trends assumption, we also performed the DID analysis comparing outcome trends during the preintervention period (i.e., comparing 2010 vs 2011) in all comparison groups.

RESULTS Patient and hospital characteristics of the 48,110 septic shock patients stratified by pre- versus postimplementation and collaborative status are displayed in Table 1. Patients and hospital characteristics stratified by high and low adherence hospitals are displayed in Supplementary Table 1 (Supplemental Digital Content 2, http://links.lww.com/CCM/B931). The range of hospital-level bundle adherence (potential range, 0–8 measures) was 2.1–6.7, with a median of 4.8 (interquartile range, 3.6–5.2). (Fig. 1) All hospitals with an average adherence of 4.8 or greater were deemed high adherence hospitals (n = 19), and those below were deemed low adherence (n = 18). Risk-adjusted in-hospital mortality rates and hospital length of stay (d) pre- and postimplementation for each hospital cohort are shown in Figures 2 and 3, respectively. Both collaborative (33.8% vs 30.5%; p < 0.001) and noncollaborative (32.9% vs 30.9%; p = 0.002) hospitals had significant differences in pre- versus postimplementation. (Fig. 2) After stratifying collaborative hospitals by bundle adherence level, high adherence hospitals had significantly reduced in-hospital mortality between pre- and postperiods (35.0% vs 29.7%; p < 0.001), whereas low adherence hospitals did not (32.2% vs 31.7%; p = 0.643). Similarly, collaborative (12.8 vs 12.0 d; p < 0.001) and noncollaborative (12.4 vs 11.8 d; p < 0.001) hospitals had statistically significant reductions in hospital length of stay. (Fig. 3) When stratified by adherence level, high adherence hospitals had a significant reduction in length of stay (13.4 vs 12.0 d; p < 0.001), whereas low adherence hospitals did not (12.1 vs 11.8 d; p = 0.118). Table 2 shows the results of the DID analysis for in-hospital mortality and hospital length of stay. There was no significant interaction between collaborative status and pre- versus postimplementation in in-hospital mortality (DID odds ratio [OR], 0.94; 95% CI, 0.87–1.01; p = 0.106) or hospital length of stay (DID estimator, –0.3 d; 95% CI, –0.7 to 0.1 d; p = 0.174), indicating no additional effect of the collaborative on improving patient outcomes compared to noncollaborative hospitals. However, high bundle adherence hospitals had a significant reduction in in-hospital mortality (DID OR, 0.85; 95% CI, 0.78–0.93; p = 0.001) and hospital length of stay (DID estimator, –0.7 d; 95% CI, –1.1 to –0.2; p = 0.005) compared to noncollaborative hospitals. Conversely, low bundle adherence hospitals had no significant changes in in-hospital mortality (DID OR, 1.07; 95% CI, 0.97–1.19; p = 0.197) or hospital length of stay (DID estimator, 0.2 d; 95% CI, –0.3 to 0.8 d; p = 0.367) compared to noncollaborative hospitals. www.ccmjournal.org

2125

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Thompson et al

Patient and Hospital Characteristics of Collaborative (n = 37) and Noncollaborative (n = 50) Hospitals Before (2010–2011) and After (2012–2013) Full Implementation of the Michigan Health & Hospital Association Keystone Sepsis Collaborative

Table 1.

Preimplementation (2010–2011) n = 22,197 Collaborative Noncollaborative n = 10,893 (49.1%) n = 11,304 (50.9%)

Variable

Postimplementation (2012–2013) n = 25,913 p

Collaborative Noncollaborative n = 12,782 (49.3%) n = 13,131 (50.7%)

p

Patient characteristics   Male gender  Age

a

  Charlson comorbidity index

a

50.3

51.4

67.6 (15.6)

66.2 (15.9)

2.8 (2.3)

2.8 (2.3)

  Insurance type

0.110 < 0.001 0.109

49.9

51.2

67.5 (15.4)

66.4 (15.8)

2.8 (2.3)

2.7 (2.3)

< 0.001

< 0.001 0.149 < 0.001

  Medicare

70.9

66.8

70.1

69.2

  Medicaid

10.6

10.8

11.4

10.1

  Private

15.7

17.3

15.9

16.9

  Other

2.8

5.2

2.6

3.7

  Admission type

0.032

< 0.001

< 0.001

  Emergency

82.6

69.3

87.0

73.2

  Urgent

11.4

23.2

9.1

22.4

  Elective

3.3

4.3

3.7

3.7

  Unknown

2.6

3.2

0.3

0.7

20.1

14.6

< 0.001

21.2

16.0

< 0.001

8.8

3.9

< 0.001

9.8

4.2

< 0.001

63.1

64.8

0.009

63.4

65.6

0.003

  Transferred patient Hospital characteristics  Rural  Teaching   Bed size

< 0.001

< 0.001

  300+ beds

56.1

77.3

57.7

78.2

  100–300 beds

34.6

17.2

33.3

17.0

9.4

5.5

9.1

5.8

   < 100 beds   Hospital Medicaid proportion

a

19.4 (7.7)

17.4 (8.1)

< 0.001

19.1 (7.7)

16.8 (7.6)

< 0.001

Patient outcomes   Hospital discharge status

< 0.001

0.001

  Died

33.8

32.9

30.5

30.9

   Discharged to home

11.2

14.2

13.4

14.1

   Discharged to extra care

45.5

45.2

46.1

46.6

9.6

7.6

10.0

8.5

   Discharged to hospice   Hospital length of stay (d), median (IQR)

10 (5–17)

9 (5–16)

0.008

9 (5–16)

8 (4–15)

0.147

IQR = interquartile range. a Mean (sd). n = 48,110 patients.

2126

www.ccmjournal.org

December 2016 • Volume 44 • Number 12

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Feature Articles

Supplemental Digital Content 4, http://links.lww.com/CCM/ B933).

DISCUSSION We used a controlled, pre-post design (DID analysis) to evaluate a multi-ICU QI collaborative employing a protocol-based resuscitation bundle to improve outcomes in septic shock patients. Our study has several important findings. First, we illustrate recent trends in septic shock outcomes from a diverse, statewide population, which saw a 2–3% absolute reduction in in-hospital mortality and a 1-day reduction in length of stay over a comparatively short 4-year period. Second, we found that participation in the KeyFigure 1. Hospital-level adherence to the protocol-based resuscitation bundle for hospitals participation in the Michigan Health & Hospital Association Keystone Sepsis collaborative (n = 37). IQR = interquartile range. stone Sepsis collaborative was not associated with additional When excluding patients who died, compared to noncol- improvements in in-hospital mortality or hospital length of laborative hospitals, collaborative hospitals did have a significant stay. By using a concurrent control group of hospitals, we were additional reduction in length of stay (DID estimator, –0.5 d; 95% able to separate out underlying secular trends in outcomes CI, –1.0 to –0.03; p = 0.038). Only high adherence hospitals had from collaborative effects, thereby estimating the independent an additional reduction in length of stay compared to noncol- impact of the collaborative. Most observational studies of prolaborative hospitals (DID estimator, –0.7 d; 95% CI, –1.4 to –0.3; tocol-based therapies use either a historical control group or p = 0.003), whereas low adherence hospitals did not (DID estino control group at all and are unable to distinguish the intermator, –0.1 d; 95% CI, –0.7 to 0.6; p = 0.801) (Supplementary vention effect from the underlying secular trend adequately Table 2, Supplemental Digital Content 3, http://links.lww.com/ (30–34). Finally, we demonstrated substantial variation in CCM/B932). After removing 2011 data from our analyses, our bundle adherence between hospitals and that hospitals with results did not change substantially. When testing for parallel high adherence to the protocol received additional improvetrends, we found no significant difference in trends during the ments in patient outcomes beyond the observed secular trends. preintervention period between any of the comparison groups, These results may help explain why simply participation in the suggesting that the assumption holds (Supplementary Table 3, QI collaborative was not sufficient to improve outcomes. The simplest explanation for our findings in high adherence hospitals is that the additional improvements in outcomes beyond concurrent trends are due to the clinical benefit of the protocol-based resuscitation bundle. Observational studies have shown that protocol-based therapies do improve patient outcomes, although results should be interpreted in light of potential selection and confounding biases (6, 30–34). Conversely, recent clinical trials have found that protocolbased therapies provided no additional benefit in outcomes compared to usual care for septic shock (7–9). However, the reported overall mortality in these trials were 25% or less, which is lower than the mortality rates reported in this study and in more recent SSC data (29%) (6). Patients in these trials may not represent the typical spectrum of septic shock patients or be comparable with earlier studies of protocol-based therapies (4, 35). Since we did not have information on patient-level Figure 2. Pre-post implementation comparisons of risk-adjusted bundle adherence, we can only speculate on the causal link in-hospital mortality, stratified by hospital collaborative participation, as well as by adherence level in collaborative hospitals. between bundle adherence and improved patient outcomes. Critical Care Medicine

www.ccmjournal.org

2127

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Thompson et al

Figure 3. Pre-post implementation comparisons of risk-adjusted hospital length of stay (in days), stratified by hospital collaborative participation, as well as by adherence level in collaborative hospitals.

However, there may be alternative explanations for why high adherence hospitals gained additional improvements in outcomes, beyond the purported clinical benefits of the resuscitation bundle. The intent of the cultural improvement component of the intervention (CUSP) was to create a collaborative and safe work environment, enhance leadership support, and increase receptiveness to process improvement, which are critical characteristics of successful QI projects (36, 37). Hospitals with high adherence to the bundle may reflect many of these characteristics. Exploring the relationship between unit culture, bundle adherence, and patient outcomes could prove especially valuable in understanding our findings. Furthermore, because limited resources and treatment complexity are common barriers to protocolbased therapies, further research should explore the role of characteristics of the ICUs, such as resources, staffing, and training (18–20). Successful implementation of QI initiatives rely on three components: the clinical context of the initiative, the facilitation of the initiative in practice, and the quality of the evidence employed by the initiative (37). The primary function of the MHA Keystone Center is to provide assistance with the first two components through cultural improvement and project management and coordination. Thus, the only unique component in each collaborative is the evidence-based practice being employed. In the Keystone ICU program, a substantial reduction in the incidence of central line-associated bloodstream infections (CLABSI) was achieved (38). This was likely due in large part to clear, high-quality evidence for prevention of CLABSI (39, 40). Conversely, the Keystone Surgery program did not show any improvement in surgical outcomes, which the authors attributed to mixed evidence on the efficacy of surgical checklists to improve outcomes (41). We believe that skepticism surrounding the clinical efficacy of protocol-based therapies for septic shock may have limited the success of the Keystone Sepsis collaborative, as illustrated by the 2128

www.ccmjournal.org

substantial hospital-level disparities in bundle adherence. Guidelines requiring placement of central venous catheters are perceived by many care providers to be invasive, timeconsuming, and providing little direct gain to the patient (5, 14). It is important to note that the SSC clinical guidelines indicate that catheter-based measures have low levels of evidence for benefit (grade C) (24). This study combined with findings from previous MHA Keystone Center collaboratives underscore the importance of high-quality evidence for interventions used in QI initiatives. Finally, it is possible that our findings are simply the result of regression to the mean. In-hospital mortality rates and length of stay in high adherence hospitals were the highest of all groups in the preimplementation phase and the lowest in the postimplementation phase. When using the DID method, it is important that expected improvements in outcomes are not associated with baseline levels (29, 42). If high adherence hospitals had relatively poorer adherence prior to the intervention, we might expect those hospitals to have greater improvements in outcomes. Since we did not have preintervention bundle adherence data, we were unable to examine how changes in bundle adherence affected outcome changes over time. This analysis would confirm that our findings were not simply a regression to the mean. There are limitations to our study to consider. First, important prognostic factors not available in the MIDB, such as infection source, serum lactate, and Acute Physiology and Chronic Health Evaluation II score, could lead to unmeasured confounding (43–45). Second, recent evidence suggests that improved recognition of severe sepsis and septic shock may increase reported case incidence (1, 10). We show similar trends in the number of septic shock cases in collaborative and noncollaborative hospitals from pre- to postimplementation, suggesting nondifferential changes in recognition and coding of septic shock between groups. Third, although the MHA intervention applied to patients treated in the ICU, the MIDB includes all discharged patients, which may include non-ICU septic shock patients. Although rare, mortality in non-ICU septic shock patients may have higher mortality compared to ICU patients (46). Finally, the MIDB did not provide data on other relevant outcomes, such as recurrent organ failure, postdischarge ambulatory status, costs, and postdischarge mortality.

CONCLUSIONS This study found that participation in this ICU-based QI collaborative was not sufficient to improve outcomes in septic shock patients beyond concurrent trends. Only hospitals with high bundle adherence had significantly greater improvements in outcomes. Future work should explore the mechanism behind the successes of high adherence hospitals. Nevertheless, substantial work remains to improve outcomes in septic shock patients. A universally accepted set of treatment guidelines supported by high-quality evidence is essential to ensure the success of future septic shock QI initiatives. December 2016 • Volume 44 • Number 12

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Feature Articles

Table 2. Results of Difference-In-Difference Analysis for In-Hospital Mortality and Hospital Length of Stay (in Days) Showing Unadjusted and Adjusted Pre-Post Comparisons and the Difference-In-Difference Estimator for the Independent Collaborative Effect Pre-Post Comparisons Patient Cohort

Unadjusted

+ Patient Characteristicsa

+ Hospital Characteristicsb

DifferenceIn-Difference Estimatorc

p

OR (95% CI) for septic shock mortality   Collaborative (n = 37)

0.86 (0.81–0.91)

0.86 (0.81–0.91)

0.86 (0.81–0.91)

0.94 (0.87–1.01)

0.106

  Noncollaborative (n = 50)

0.91 (0.86–0.96)

0.91 (0.87–0.97)

0.92 (0.87–0.97)

Reference

—

  High adherence (n = 19)

0.79 (0.73–0.84)

0.78 (0.73–0.84)

0.78 (0.73–0.84)

0.85 (0.78–0.93)

0.001

  Low adherence (n = 18)

0.97 (0.89–1.06)

0.98 (0.90–1.07)

0.98 (0.90–1.07)

1.07 (0.97–1.19)

0.197

  Noncollaborative (n = 50)

0.91 (0.86–0.96)

0.91 (0.87–0.97)

0.92 (0.87–0.97)

Reference

—

Absolute change in length of stay (95% CI) (d)   Collaborative (n = 37)

–0.9 (–1.2 to –0.6)

–0.9 (–1.2 to –0.6)

–0.9 (–1.2 to –0.6)

–0.3 (–0.7 to 0.1)

0.174

  Noncollaborative (n = 50)

–0.7 (–1.0 to –0.4)

–0.6 (–0.9 to –0.3)

–0.6 (–0.9 to –0.3)

Reference

—

  High adherence (n = 19)

–1.3 (–1.7 to –1.0)

–1.3 (–1.7 to –0.9)

–1.3 (–1.6 to –0.9)

–0.7 (–1.1 to –0.2)

0.005

  Low adherence (n = 18)

–0.3 (–0.7 to 0.2)

–0.3 (–0.8 to 0.1)

–0.4 (–0.8 to 0.1)

0.2 (–0.3 to 0.8)

0.367

  Noncollaborative (n = 50)

–0.7 (–1.0 to –0.4)

–0.6 (–0.9 to –0.3)

–0.6 (–0.9 to –0.3)

Reference

—

OR = odds ratio. a Adjusted for age, gender, insurance status, Charlson comorbidity index, type of admission, and transfer status. b Adjusted for patient characteristics plus rural versus urban, teaching status, bed size, and hospital Medicaid proportion. c Difference-in-difference estimator is the comparison of pre-post differences between patient cohorts (collaborative vs noncollaborative, high/low adherence vs noncollaborative) and represents the independent effect of the Michigan Health & Hospital Association Keystone Sepsis collaborative.

REFERENCES

1. Gaieski DF, Edwards JM, Kallan MJ, et al: Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med 2013; 41:1167–1174 2. Kaukonen KM, Bailey M, Suzuki S, et al: Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000-2012. JAMA 2014; 311:1308–1316 3. Lagu T, Rothberg MB, Shieh MS, et al: Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007. Crit Care Med 2012; 40:754–761 4. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy Collaborative Group: Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001; 345:1368–1377 5. Dellinger RP, Levy MM, Rhodes A, et al; Surviving Sepsis Campaign Guidelines Committee including The Pediatric Subgroup: Surviving Sepsis Campaign: International guidelines for management of severe sepsis and septic shock, 2012. Intensive Care Med 2013; 39:165–228 6. Levy MM, Rhodes A, Phillips GS, et al: Surviving Sepsis Campaign: Association between performance metrics and outcomes in a 7.5-year study. Crit Care Med 2015; 43:3–12 7. Peake SL, Delanay A, Bailey M, et al; The ARISE Investigators; ANZICS Clinical Trials Group: Goal-directed resuscitation for patients with early septic shock. N Engl J Med 2014; 371:1496–1506

Critical Care Medicine

8. Yealy DM, Kellum JA, Huang DT, et al; The ProcESS Investigators: A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014; 370:1683–1693 9. Mouncey PR, Osborn TM, Power GS, et al; ProMISe Trial Investigators: Trial of early, goal-directed resuscitation for septic shock. N Engl J Med 2015; 372:1301–1311 10. Rhee C, Gohil S, Klompas M: Regulatory mandates for sepsis care– reasons for caution. N Engl J Med 2014; 370:1673–1676 11. Gao F, Melody T, Daniels DF, et al: The impact of compliance with 6-hour and 24-hour sepsis bundles on hospital mortality in patients with severe sepsis: A prospective observational study. Crit Care 2005; 9:R764–R770 12. Casserly B, Baram M, Walsh P, et al: Implementing a collaborative protocol in a sepsis intervention program: Lessons learned. Lung 2011; 189:11–19 13. Crowe CA, Mistry CD, Rzechula K, et al: Evaluation of a modified early goal-directed therapy protocol. Am J Emerg Med 2010; 28:689–693 14. O’Neill R, Morales J, Jule M: Early goal-directed therapy (EGDT) for severe sepsis/septic shock: Which components of treatment are more difficult to implement in a community-based emergency department? J Emerg Med 2012; 42:503–510 15. Jones AE, Shapiro NI, Roshon M: Implementing early goal-directed therapy in the emergency setting: The challenges and experiences of www.ccmjournal.org

2129

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

Thompson et al translating research innovations into clinical reality in academic and community settings. Acad Emerg Med 2007; 14:1072–1078 16. Reade MC, Huang DT, Bell D, et al; British Association for Emergency Medicine; UK Intensive Care Society; UK Society for Acute Medicine; Australasian Resuscitation in Sepsis Evaluation Investigators; Protocolized Care for Early Septic Shock Investigators: Variability in management of early severe sepsis. Emerg Med J 2010; 27:110–115 17. Turi SK, Von Ah D: Implementation of early goal-directed therapy for septic patients in the emergency department: A review of the literature. J Emerg Nurs 2013; 39:13–19 18. Carlbom DJ, Rubenfeld GD: Barriers to implementing protocolbased sepsis resuscitation in the emergency department–results of a national survey. Crit Care Med 2007; 35:2525–2532 19. Burney M, Underwood J, McEvoy S, et al: Early detection and treatment of severe sepsis in the emergency department: Identifying barriers to implementation of a protocol-based approach. J Emerg Nurs 2012; 38:512–517 20. Jones AE, Kline JA: Use of goal-directed therapy for severe sepsis and septic shock in academic emergency departments. Crit Care Med 2005; 33:1888–1889; author reply 1889 21. Pronovost PJ, Berenholtz SM, Goeschel CA, et al: Creating high reliability in health care organizations. Health Serv Res 2006; 41:1599–1617 22. Johns Hopkins Medicine: Center for Innovation in Quality Patient Care: The Comprehensive Unit-based Safety Program (CUSP). Available at: http://www.hopkinsmedicine.org/innovation_quality_patient_care/areas_ expertise/improve_patient_safety/cusp/. Accessed June 30, 2014 23. Pronovost P, Weast B, Rosenstein B, et al: Implementing and validating a comprehensive unit-based safety program. J Patient Saf 2005; 1:33–40 24. Dellinger RP, Levy MM, Carlet JM, et al; International Surviving Sepsis Campaign Guidelines Committee; American Association of Critical-Care Nurses; American College of Chest Physicians; American College of Emergency Physicians; Canadian Critical Care Society; European Society of Clinical Microbiology and Infectious Diseases; European Society of Intensive Care Medicine; European Respiratory Society; International Sepsis Forum; Japanese Association for Acute Medicine; Japanese Society of Intensive Care Medicine; Society of Critical Care Medicine; Society of Hospital Medicine; Surgical Infection Society; World Federation of Societies of Intensive and Critical Care Medicine: Surviving Sepsis Campaign: International guidelines for management of severe sepsis and septic shock: 2008. Crit Care Med 2008; 36:296–327 25. Data Koala: Product Information - Michigan Inpatient Database (MIDB), 2014. Available at: http://www.datakoala.com/productinfo. htm. Accessed June 17, 2015 26. Deyo RA, Cherkin DC, Ciol MA: Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992; 45:613–619 27. Quan H, Sundararajan V, Halfon P, et al: Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005; 43:1130–1139 28. Dimick JB, Ryan AM: Methods for evaluating changes in health care policy: The difference-in-differences approach. JAMA 2014; 312:2401–2402 29. Ryan AM, Burgess JF Jr, Dimick JB: Why we should not be indifferent to specification choices for difference-in-differences. Health Serv Res 2015; 50:1211–1235

2130

www.ccmjournal.org

30. Castellanos-Ortega A, Suberviola B, García-Astudillo LA, et al: Impact of the Surviving Sepsis Campaign protocols on hospital length of stay and mortality in septic shock patients: Results of a three-year followup quasi-experimental study. Crit Care Med 2010; 38:1036–1043 31. Levy MM, Dellinger RP, Townsend SR, et al: The Surviving Sepsis Campaign: Results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med 2010; 36:222–231 32. Micek ST, Roubinian N, Heuring T, et al: Before-after study of a standardized hospital order set for the management of septic shock. Crit Care Med 2006; 34:2707–2713 33. Ferrer R, Artigas A, Levy MM, et al; Edusepsis Study Group: Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA 2008; 299:2294–2303 34. Nguyen HB, Corbett SW, Steele R, et al: Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit Care Med 2007; 35:1105–1112 35. Seymour CW, Rosengart MR: Septic Shock: Advances in Diagnosis and Treatment. JAMA 2015; 314:708–717 36. Kaplan HC, Brady PW, Dritz MC, et al: The influence of context on quality improvement success in health care: A systematic review of the literature. Milbank Q 2010; 88:500–559 37. Helfrich CD, Damschroder LJ, Hagedorn HJ, et al: A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implement Sci 2010; 5:82 38. Pronovost P, Needham D, Berenholtz S, et al: An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006; 355:2725–2732 39. Casey AL, Mermel LA, Nightingale P, et al: Antimicrobial central venous catheters in adults: A systematic review and meta-analysis. Lancet Infect Dis 2008; 8:763–776 4 0. Timsit JF, Schwebel C, Bouadma L, et al; Dressing Study Group: Chlorhexidine-impregnated sponges and less frequent dressing changes for prevention of catheter-related infections in critically ill adults: A randomized controlled trial. JAMA 2009; 301:1231–1241 41. Reames BN, Krell RW, Campbell DA Jr, et al: A checklist-based intervention to improve surgical outcomes in Michigan: Evaluation of the Keystone Surgery program. JAMA Surg 2015; 150:208–215 42. Chay KY, McEwan PJ, Urquiola M: The central role of noise in evaluating interventions that use test scores to rank schools. Am Econ Rev 2005; 95:1237–1258 43. Leligdowicz A, Dodek PM, Norena M, et al; Co-operative Antimicrobial Therapy of Septic Shock Database Research Group: Association between source of infection and hospital mortality in patients who have septic shock. Am J Respir Crit Care Med 2014; 189:1204–1213 44. Nguyen HB, Rivers EP, Knoblich BP, et al: Early lactate clearance is associated with improved outcome in severe sepsis and septic shock. Crit Care Med 2004; 32:1637–1642 45. Knaus WA, Draper EA, Wagner DP, et al: APACHE II: A severity of disease classification system. Crit Care Med 1985; 13:818–829 46. Esteban A, Frutos-Vivar F, Ferguson ND, et al: Sepsis incidence and outcome: Contrasting the intensive care unit with the hospital ward. Crit Care Med 2007; 35:1284–1289

December 2016 • Volume 44 • Number 12

Copyright © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.