Airway Registry and Training Curriculum Improve Intubation Outcomes in the Intensive Care Unit

Joshua Malo MD¹

Cameron Hypes MD²

Bhupinder Natt MBBS¹

Elaine Cristan MD¹

Jeremy Greenberg MD¹

Katelin Morrissette MD¹

Linda Snyder MD¹

James Knepler MD¹

John Sakles MD²

Kenneth Knox MD¹

Jarrod Mosier MD²

¹ Department of Medicine, University of Arizona College of Medicine, Tucson, AZ

² Department of Emergency Medicine, University of Arizona College of Medicine, Tucson, AZ

Abstract

Background: Intubation in critically ill patients remains a highly morbid procedure, and the optimal approach is unclear. We sought to improve the safety of intubation by implementing a simulation curriculum and monitoring performance with an airway registry. 

Methods and Methods: This is a prospective, single-center observational study of all intubations performed by the medical intensive care unit (ICU) team over a five-year period. All fellows take part in a simulation curriculum to improve airway management performance and minimize complications. An airway registry form is completed immediately after each intubation to capture relevant patient, operator, and procedural data.  

Results: Over a five-year period, the medical ICU team performed 1411 intubations. From Year 1 to Year 5, there were significant increases in first-attempt success (72.6 vs. 88.0%, p<0.001), use of video laryngoscopy (72.3 vs. 93.5%, p<0.001), and use of neuromuscular blocking agents (73.5 vs. 88.4%, p<0.001). There were concurrent decreases in rates of desaturation (25.6 vs. 17.1%, p=0.01) and esophageal intubations (5 vs. 1%, p=0.009). Low rates of hypotension (8.3%) and cardiac arrest (0.6%) were also observed.

Conclusions: The safety of intubation in critically ill patients can be markedly improved through joint implementation of an airway registry and simulation curriculum.

Introduction

Airway management is one of the highest risk procedures that can be performed in the intensive care unit (ICU). Despite technologic advances in methods for performing intubation, recent studies continue to report frequent adverse events associated with tracheal intubation, and complications occur in up to 40% of procedures (1-3). Even in the absence of anatomic predictors of a difficult airway, critically ill patients are particularly vulnerable to desaturation, hemodynamic instability, and cardiac arrest due to poor physiologic reserve (4, 5). Repeated or prolonged intubation attempts exhaust any physiologic reserve these patients may have, leading to more frequent adverse outcomes (6). Thus, maximizing first attempt success without an adverse event is the goal for airway management in this high-risk population (7, 8).

Much of the clinical practice regarding airway management in the ICU has been extrapolated from studies performed during elective intubations in the operating room (4). In recent years, there has been a greater focus on management strategies and outcomes in critically ill patients in the emergency department (ED) and ICU (3, 9, 10). In 2012, we initiated a comprehensive airway management quality improvement program to measure variables related to airway management in the ICU and identify targeted opportunities for intervention to improve outcomes (11). We first established a prospectively collected registry of all intubations performed in the medical ICU. After evaluation of the first year of data, a simulation-based curriculum for the pulmonary and critical care fellows was developed with a focus on identifying high-risk features, minimizing adverse events, and maximizing first-attempt success. Lastly, research questions were evaluated periodically to identify targeted opportunities for improvement. This paper will describe the outcomes after the first 5 years of our program.

Materials and Methods

This is a prospective single-center observational study of all intubations performed in the medical ICU from January 1, 2012 to December 31, 2016. The study has been granted an exemption from full review and is approved by the University of Arizona Institutional Review Board. The primary outcome of interest was first attempt success, while secondary outcomes included adverse events, drug and device selection, and method of preoxygenation.

This study took place at a large academic medical center with 20+ bed medical ICU. A medical ICU team consisting of an attending intensivist, a pulmonary/critical care or emergency medicine/critical care fellow, and internal medicine, emergency medicine, and occasionally family medicine residents assumes primary management of all patients admitted to the medical ICU service. All patients admitted to the ICU undergoing airway management by the medical ICU team were included in the study.

We have maintained a continuous quality improvement (CQI) database for all episodes of airway management performed by our medical intensive care teams since January 1, 2012. After each intubation, the operators record data pertinent to the procedure, including difficult airway characteristics, drug and device selection, and number of attempts, using a standardized form. The study primary investigator crosschecked a report generated by the electronic health record against the database to ensure forms were completed for all intubations. Forms were reviewed for completeness and internal consistency. Inconsistent or absent data were resolved by interview of the operator. The variables captured in the form have been previously described (11) and are adjusted occasionally to evaluate new variables of interest.

Our Pulmonary and Critical Care Medicine (PCCM) and Critical Care Medicine (CCM) fellowship programs implemented an 11-month, simulation-based airway management curriculum beginning on July 1, 2013. The curriculum is designed to improve situational awareness in the peri-intubation period as well as to emphasize techniques that will optimize chances of first-attempt success while minimizing complications. The general outline for the simulations has been previously described in detail (11). Briefly, the curriculum involves clinical scenarios of varying and generally progressive complexity, each of which is meant to emphasize certain aspects of airway management. As trainees progress, the curriculum emphasizes the identification and mitigation of factors that may decrease the likelihood of first-attempt success and increase the likelihood of complications. The annual fellowship complement includes 14 Pulmonary and Critical Care Medicine fellows and 2 Critical Care Medicine fellows. All fellows participate in the curriculum, which is updated to include recent advances in airway management from the literature and analysis of our own airway registry. A debriefing session following each simulation is used to emphasize specific learning points for the approach to airway management.

Statistical Analysis

Descriptive statistics were calculated for measured variables as means and standard deviations, medians and interquartile ranges (IQR), or proportions as appropriate. Categorical variables were compared using Fisher’s exact test. Comparisons between Year 1 and Year 5 were performed using the Two-Sample Test of Proportions. Categorical variables with multiple groups, such as preoxygenation, Operator PGY, and Device were evaluated with the test for trend using the likelihood ratio test. All statistical analyses were performed with Stata Version 14 (StataCorp, College Station, TX).

Results

During the 60-month study period, there were 1411 intubations performed. The patient and operator characteristics are shown in Table 1 and Table 2, respectively.

Table 1. Patient characteristics.

^aSome DACs added over time. Limited mouth opening and secretions added after the first 8 months of data collection.

Table 2. First Operator Characteristics.

During the course of the study, there was no significant change in patient age or gender, the presence of difficult airway characteristics, starting saturation, or percentage of patients intubated after failing noninvasive positive pressure ventilation (NIPPV). There was a trend for decreased intubations resulting from failed extubation. The overall characteristics of intubation attempts are described in Table 3.

Table 3. Intubation characteristics.

There was a significant increase in the number of intubations performed by PCCM operators after the first year of the study (Year 1-5 difference +19%, p<0.001) accompanied by a decrease in intubations performed by internal medicine (Year 1-5 difference -9%, p=0.006) and emergency medicine residents (Year 1-5 difference -10%, p=0.003). Likewise, there was an increase in intubations performed by PGY 4 (Year 1-5 difference +13%, p=0.002) and PGY 6 (Year 1-5 difference +11%, p<0.001) operators with a concurrent decrease in those performed by PGY 2 (Year 1-5 difference -16%, p<0.001) and PGY 3 (Year 1-5 difference -8%, p=0.004) operators.

First-attempt success (FAS) occurred in 80.7% of intubations performed during the study period. The FAS rate increased linearly throughout the study period, with FAS of 72.6% in the first year and 88.0 % in the final year when looking at all operators (p<0.001) (Table 4, Figure 1, next page). 

There was a significant increase in the number of intubations performed by PCCM operators after the first year of the study (Year 1-5 difference +19%, p<0.001) accompanied by a decrease in intubations performed by internal medicine (Year 1-5 difference -9%, p=0.006) and emergency medicine residents (Year 1-5 difference -10%, p=0.003). Likewise, there was an increase in intubations performed by PGY 4 (Year 1-5 difference +13%, p=0.002) and PGY 6 (Year 1-5 difference +11%, p<0.001) operators with a concurrent decrease in those performed by PGY 2 (Year 1-5 difference -16%, p<0.001) and PGY 3 (Year 1-5 difference -8%, p=0.004) operators.

First-attempt success (FAS) occurred in 80.7% of intubations performed during the study period. The FAS rate increased linearly throughout the study period, with FAS of 72.6% in the first year and 88.0 % in the final year when looking at all operators (p<0.001) (Table 4, Figure 1).

Table 4. Outcomes.

Figure 1. First-attempt success and complications over time.

For patients intubated by fellows only, FAS increased from 77% to 92% over the 5-year period (p<0.001).

During the entire study period, at least one complication occurred in 28.7% of intubations. The incidence of complications decreased throughout the first 48 months but increased slightly in the final 12 months of the study, driven primarily by an increase in hypotension (Table 4, Figure 2).

Figure 2. Neuromuscular blocking agent (NMBA) use, video laryngoscopy (VL) use, and occurrence of esophageal intubations, desaturation, and hypotension over time.

There was a decrease in the rate of desaturation from the first year to the final year of the study (25.6% to 17.1%, p=0.01). Esophageal intubations also decreased significantly over this time (5% to 1%, p=0.009). Hypotension and cardiac arrest occurred in 8.3% and 0.6% of intubations, respectively, during the entire study period.

There was a trend towards decreased use of midazolam and propofol throughout the study period while the use of etomidate tended to increase, although these changes were not significant (Table 3). A neuromuscular blocking agent (NMBA) was used in 77.4% of intubations during the study period, increasing from the first year to the final year (73.5% to 88.4%, p<0.001), driven primarily by an increase in the use of rocuronium.

There was a significant transition from the use of direct laryngoscopy (DL) to video laryngoscopy (VL) over the course of the study (p<0.001). DL was chosen as the first approach in 22.0% of intubations in the first year and only 2.9% in the final year. Conversely, the use any form of VL on the first attempt increased from 72.3% of intubations in the first year to 93.5% in the final year. Flexible fiber optic intubation was used infrequently during the entire study period, being the first device used in 4.3% of intubations.

Various methods of preoxygenation were used throughout the study period with some form of preoxygenation occurring in 97.5% of intubations. From the first year to the final year of the study, the use of bag-valve-mask (BVM) ventilation tended to decrease (30% to 12.2%) with a concurrent trend in increasing use of NIPPV for preoxygenation (19.4% to 29.7%).

Discussion

Our experience demonstrates that utilization of a comprehensive approach to airway management including an ongoing simulation-based training curriculum and CQI database is associated with an improved first-attempt success rate for the intubation of critically ill patients. This was accompanied by changes in approach to airway management, with increased use of VL and NMBA on the first attempt, as well as an increased proportion of airways being managed by more experienced operators.

While some of the observed improvement in FAS may be attributed to more experienced operators managing the airway on the first attempt, the sharp increase in fellow-level operators after implementation of the curriculum may point to increased fellow confidence or increased recognition of high-risk patients. Furthermore, as adjunctive strategies such as ramp positioning (12-14) and apneic oxygenation (15, 16) have become increasingly recognized as potentially beneficial, a continuous training curriculum provides opportunities for evaluating trainees’ knowledge of these techniques and reinforcing their incorporation into airway management.

We have previously reported on the impact of a simulation-based curriculum on operator confidence, first-attempt success, and procedural complications (11). The combination of this curriculum with a CQI database has a marked effect on the approach to management of these patients. Strategies presented and employed in the curriculum have been informed by previous reports from our database. For example, after demonstrating improved first-attempt success with the use of neuromuscular blockade (17) and video laryngoscopy (18), the didactic portion of our curriculum incorporated these findings, which were rapidly used with increasing frequency in our intensive care unit. The integration of the curriculum and CQI database facilitates adoption of best practices, leading to a significant improvement of first-attempt success rate over a relatively short time span. The continued improvement over the course of five years is likely due to the incorporation of the practices above during this time.

Tools traditionally used for predicting difficulty of airway management have focused primarily on characteristics of an anatomically difficult intubation (19, 20). More recently, there has been an expanded focus on physiologic characteristics that may lead to complications and decreased success of intubation. However, currently available instruments for ICU patients, such as the MACOCHA score, continue to put heavy emphasis on anatomic factors and are not validated for the use of VL (21). We have found difficult airway characteristics associated with decreased FAS in the setting of VL and have focused efforts at minimizing their impact (22). In our population, we noted a consistent improvement in Cormack-Lehane grade and percentage of glottic opening (POGO) score, despite a high prevalence of anatomic difficult airway characteristics. We have also noted a significant decrease in desaturations, esophageal intubations, and a trend towards decreased overall complications. In comparison to other studies of intubation complications in critically ill patients, we found generally lower rates of esophageal intubation (1, 2, 6) and similar (10) or lower rates of desaturation (1, 2) (Table 5).

Table 5. Incidence of complications in the published literature.

Our rate of hypotension is fairly low relative to several other studies (2, 10, 23) despite a fairly inclusive definition (administration of fluid or phenylephrine bolus, initiation or increase of vasopressor infusion). Moreover, cardiac arrest was extremely uncommon in our population, occurring in 0.57% of intubations.

Data regarding optimal approach have been controversial, and randomized trial results do not always coincide with observational studies. Although randomized controlled trials have called into question the benefit of VL (24-27), there are several important limitations to each of these studies to consider when interpreting the comparison between DL and VL. In some, patients were excluded either directly (25) or indirectly (24, 26) for a history of a difficult intubation or anticipated difficult intubation. The use of endotracheal tubes without a stylet may have also influenced outcomes (27).

Our experience is a pragmatic example of the effect of device selection on first attempt success in that we have >1400 patients, operators with varying experience, and have no patients excluded because of potential difficulty. Thus, while randomized trials may be ideal, they are costly and time-consuming and may delay identification and implementation of best practices. The FAS rate in our cohort in its first year was similar to that observed in several of these trials but improved substantially over the 5-year period. One reason for the improved FAS in our study may be the continuous simulation-based training with a focus on video laryngoscopy as the first technique of choice for the majority of airways. In comparison to other widely cited studies of airway management in critically ill patients (1, 21), our cohort demonstrated a very low incidence of difficult airways, only 2% in the final year, despite a similar presence of difficult airway characteristics. This may be an effect of the training program suggesting that perhaps airway training with a global view of airway management focusing on increasing FAS and reducing complications is even more important than equipment considerations.

Our study has several limitations. The single-center, observational nature of this study makes it at risk of bias despite attempts to identify and control for factors that may influence the results. Data forms were completed by the operator, introducing potential for reporting bias, although attempts to minimize this were made by intermittent correlation with the medical record. Although FAS is an accepted outcome for studies evaluating intubation strategies, data regarding mortality or late morbidity were not captured. The increase in hemodynamic complications in the final year is of interest, but data regarding this complication and its consequences were limited and should be a focus of future research. Despite these limitations, the consistent improvement in FAS and low incidence of difficult airways in the final year of the study warrant serious consideration of these findings.

Conclusion

We have found that a comprehensive strategy employing a simulation-based curriculum and continuous quality improvement database was associated with significant improvements in first-attempt success at intubation in critically ill patients throughout the 5-year study period. We suggest that wider adoption of this practice could vastly improve the safety of intubation in this high-risk patient population.

References

1. Griesdale DE, Bosma TL, Kurth T, Isac G, Chittock DR. Complications of endotracheal intubation in the critically ill. Intensive Care Med. 2008;34(10):1835-42. [CrossRef] [PubMed]
2. Jaber S, Amraoui J, Lefrant JY, Arich C, Cohendy R, Landreau L, et al. Clinical practice and risk factors for immediate complications of endotracheal intubation in the intensive care unit: a prospective, multiple-center study. Crit Care Med. 2006;34(9):2355-61. [CrossRef] [PubMed]
3. Lapinsky SE. Endotracheal intubation in the ICU. Crit Care. 2015;19:258. [CrossRef] [PubMed]
4. Mort TC. The incidence and risk factors for cardiac arrest during emergency tracheal intubation: a justification for incorporating the ASA Guidelines in the remote location. J Clin Anesth. 2004;16(7):508-16. [CrossRef] [PubMed]
5. Mosier JM, Joshi R, Hypes C, Pacheco G, Valenzuela T, Sakles JC. The Physiologically Difficult Airway. West J Emerg Med. 2015;16(7):1109-17. [CrossRef] [PubMed]
6. Mort TC. Emergency tracheal intubation: complications associated with repeated laryngoscopic attempts. Anesth Analg. 2004;99(2):607-13, table of contents.  [CrossRef] [PubMed]
7. Hypes C, Sakles J, Joshi R, Greenberg J, Natt B, Malo J, et al. Failure to achieve first attempt success at intubation using video laryngoscopy is associated with increased complications. Intern Emerg Med. 2017 Dec;12(8):1235-43. [CrossRef] [PubMed]
8. Park L, Zeng I, Brainard A. Systematic review and meta-analysis of first-pass success rates in emergency department intubation: Creating a benchmark for emergency airway care. Emerg Med Australas. 2017;29(1):40-7. [CrossRef] [PubMed]
9. Simpson GD, Ross MJ, McKeown DW, Ray DC. Tracheal intubation in the critically ill: a multi-centre national study of practice and complications. Br J Anaesth. 2012;108(5):792-9. [CrossRef] [PubMed]
10. Smischney NJ, Seisa MO, Heise KJ, Busack KD, Loftsgard TO, Schroeder DR, et al. Practice of Intubation of the Critically Ill at Mayo Clinic. J Intensive Care Med. 2017:885066617691495. [CrossRef] [PubMed]
11. Mosier JM, Malo J, Sakles JC, Hypes CD, Natt B, Snyder L, et al. The impact of a comprehensive airway management training program for pulmonary and critical care medicine fellows. A three-year experience. Ann Am Thorac Soc. 2015;12(4):539-48. [CrossRef] [PubMed]
12. Khandelwal N, Khorsand S, Mitchell SH, Joffe AM. Head-Elevated Patient Positioning Decreases Complications of Emergent Tracheal Intubation in the Ward and Intensive Care Unit. Anesth Analg. 2016;122(4):1101-7. [CrossRef] [PubMed]
13. Ramkumar V, Umesh G, Philip FA. Preoxygenation with 20 masculine head-up tilt provides longer duration of non-hypoxic apnea than conventional preoxygenation in non-obese healthy adults. J Anesth. 2011;25(2):189-94. [CrossRef] [PubMed]
14. Turner JS, Ellender TJ, Okonkwo ER, Stepsis TM, Stevens AC, Sembroski EG, et al. Feasibility of upright patient positioning and intubation success rates at two academic emergency departments. Am J Emerg Med. 2017. [CrossRef] [PubMed]
15. Mosier JM, Hypes CD, Sakles JC. Understanding preoxygenation and apneic oxygenation during intubation in the critically ill. Intensive Care Med. 2017;43(2):226-8. [CrossRef] [PubMed]
16. Sakles JC, Mosier JM, Patanwala AE, Arcaris B, Dicken JM. First Pass Success Without Hypoxemia Is Increased With the Use of Apneic Oxygenation During Rapid Sequence Intubation in the Emergency Department. Acad Emerg Med. 2016;23(6):703-10. [CrossRef] [PubMed]
17. Mosier JM, Sakles JC, Stolz U, Hypes CD, Chopra H, Malo J, et al. Neuromuscular blockade improves first-attempt success for intubation in the intensive care unit. A propensity matched analysis. Ann Am Thorac Soc. 2015;12(5):734-41. [CrossRef] [PubMed]
18. Hypes CD, Stolz U, Sakles JC, Joshi RR, Natt B, Malo J, et al. Video Laryngoscopy Improves Odds of first-attempt success at intubation in the intensive care unit. A propensity-matched analysis. Ann Am Thorac Soc. 2016;13(3):382-90. [CrossRef] [PubMed]
19. Mallampati SR, Gatt SP, Gugino LD, Desai SP, Waraksa B, Freiberger D, et al. A clinical sign to predict difficult tracheal intubation: a prospective study. Can Anaesth Soc J. 1985;32(4):429-34. [CrossRef] [PubMed]
20. Wilson ME, Spiegelhalter D, Robertson JA, Lesser P. Predicting difficult intubation. Br J Anaesth. 1988;61(2):211-6. [CrossRef] [PubMed]
21. De Jong A, Molinari N, Terzi N, Mongardon N, Arnal JM, Guitton C, et al. Early identification of patients at risk for difficult intubation in the intensive care unit: development and validation of the MACOCHA score in a multicenter cohort study. Am J Respir Crit Care Med. 2013;187(8):832-9. [CrossRef] [PubMed]
22. Joshi R, Hypes CD, Greenberg J, Snyder L, Malo J, Bloom JW, et al. Difficult airway characteristics associated with first-attempt failure at intubation using video laryngoscopy in the intensive care unit. Ann Am Thorac Soc. 2017;14(3):368-75. [CrossRef] [PubMed]
23. Perbet S, De Jong A, Delmas J, Futier E, Pereira B, Jaber S, et al. Incidence of and risk factors for severe cardiovascular collapse after endotracheal intubation in the ICU: a multicenter observational study. Crit Care. 2015;19:257. [CrossRef] [PubMed]
24. Driver BE, Prekker ME, Moore JC, Schick AL, Reardon RF, Miner JR. Direct versus video laryngoscopy using the c-mac for tracheal intubation in the emergency department, a randomized controlled trial. Acad Emerg Med. 2016;23(4):433-9. [CrossRef] [PubMed]
25. Griesdale DE, Chau A, Isac G, Ayas N, Foster D, Irwin C, et al. Video-laryngoscopy versus direct laryngoscopy in critically ill patients: a pilot randomized trial. Can J Anaesth. 2012;59(11):1032-9. [CrossRef] [PubMed]
26. Janz DR, Semler MW, Lentz RJ, Matthews DT, Assad TR, Norman BC, et al. Randomized trial of video laryngoscopy for endotracheal intubation of critically ill adults. Crit Care Med. 2016;44(11):1980-7. [CrossRef] [PubMed]
27. Lascarrou JB, Boisrame-Helms J, Bailly A, Le Thuaut A, Kamel T, Mercier E, et al. Video laryngoscopy vs direct laryngoscopy on successful first-pass orotracheal intubation among ICU patients: A randomized clinical trial. JAMA. 2017;317(5):483-93. [CrossRef] [PubMed]

Cite as: Malo J, Hypes C, Natt B, Cristan E, Greenberg J, Morrissette K, Snyder L, Knepler J, Sakles J, Knox K, Mosier J. Airway registry and training curriculum improve intubation outcomes in the intensive care unit. Southwest J Pulm Crit Care. 2018;16(4):212-23. doi: https://doi.org/10.13175/swjpcc037-18 PDF 

Clinical Performance Of An Automated Systemic Inflammatory Response Syndrome (SIRS) / Organ Dysfunction Alert: A System-Based Patient Safety Project

Robert A Raschke, MD, MS

Huw Owen-Reece, MBBS

Hargobind Khurana, MD 

Robert H Groves Jr, MD

Steven C Curry, MD

Mary Martin, PharmD

Brenda Stoffer

Suresh Uppalapu, MD 

Heemesh Seth, DO

Nithya Menon, MD 

Banner Good Samaritan Medical Center, Phoenix Arizona

Abstract

Objective: We have employed our electronic medical record (EMR) in an effort to identify patients at the onset of severe sepsis through an automated analysis that identifies simultaneous occurrence of systemic inflammatory response syndrome (SIRS) and organ dysfunction. The purpose of this study was to determine the positive predictive value of this alert for severe sepsis and other important outcomes in hospitalized adults.

Design: Prospective cohort.

Setting: Banner Good Samaritan Medical Center, Phoenix AZ

Patients: Forty adult inpatients who triggered alert logic within our EMR indicating simultaneous occurrence of SIRS and organ dysfunction.

Interventions: Interview of bedside nurse and chart review within six hours of alert firing to determine the clinical event that triggered each alert.

Results: Eleven of 40 patients (28%) had a major clinical event (immediately life-threatening illness) associated with the alert firing. Severe sepsis or septic shock accounted for four of these – yielding a positive predictive value of 0.10 (95%CI: 0.04-0.23) of the alert for detection of severe sepsis. The positive predictive value of the alert for detection of major clinical events was 0.28 (95%CI: 0.16-0.43), and for detecting either a major or minor clinical event was 0.45 (95%CI: 0.31-0.60). Twenty-two of 40 patients (55%) experienced a false alert.

Conclusions: Our first-generation SIRS/organ dysfunction alert has a low positive predictive value for severe sepsis, and generates many false alerts, but shows promise for the detection of acute clinical events that require immediate attention. We are currently investigating refinements of our automated alert system which we believe have potential to enhance patient safety.

Introduction

Severe sepsis is defined as systemic inflammatory response syndrome (SIRS) of infectious etiology with secondary organ dysfunction. It is estimated that 750,000 patients suffer severe sepsis annually in the United States - 3 cases per 1000 population (1). Mortality has fallen over the past several decades, but ranges from 20-30% in recent studies (1,3). Results of recent treatment trials for severe sepsis are consistent with the hypothesis that early diagnosis and treatment are important (2,3), but reliable systems for early recognition of severe sepsis in hospitalized patients are not widely available.  

We have sought to improve patient safety at our institution by using our integrated electronic medical record (EMR) to identify patients at the onset of severe sepsis through a logic algorithm that analyzes vital signs and laboratory data. This logic function identifies patients with simultaneous systemic inflammatory response syndrome (SIRS) and organ dysfunction, but cannot distinguish whether an acute infection is the cause of these findings. The purpose of this study was to determine what clinical events – infectious or non-infectious - actually cause the vital sign and laboratory changes that trigger this alert, and what the positive predictive valve of the alert is for detecting the onset of severe sepsis in hospitalized adult patients.

Methods

This was a prospective cohort study carried out at Banner Good Samaritan Medical Center – a 700-bed University-affiliated teaching hospital in Phoenix AZ. It was part of an ongoing quality improvement project and was thereby exempted from IRB approval. The SIRS / organ dysfunction alert logic was developed at Banner Health using Cerner Discern Expert^®, Cerner Corporation, North Kansas City MO, USA. The logic function monitored the EMR for standard SIRS criteria and laboratory evidence of organ dysfunction with thresholds consistent with standard definition of severe sepsis (Table 1) (4,5).

Table 1. Specific criteria for the logic function of our SIRS/organ failure alert. 

When any single criterion for SIRS was met, the program searched the prior 6 hours for the most recent vital signs, and the prior 30 hours for the most recent white blood cell count. If a second SIRS criterion was met, the program identified the patient as exhibiting SIRS, but did not trigger an alert. When any single laboratory criterion for organ dysfunction was met (table 1), the program identified the patient as suffering organ dysfunction. If criteria for SIRS and organ dysfunction overlap in any 8 hour window, the alert fired, triggering a real-time notification in the Cerner Millenium^® EMR alerting clinicians to the possibility of severe sepsis or septic shock. The alert has been in clinical application since 2010. 

We sampled 40 non-consecutive inpatients in the first three months of 2014 by a nonrandom method blinded to the patient’s clinical condition. On days of data collection, all alerts that had fired within the prior 6 hours were reviewed, regardless of patient location or diagnosis. The patient bedside was visited by a physician researcher during the six-hour window after alert firing and the nurse interviewed in order to determine the circumstances that caused the alert to fire. The patient might be briefly examined if necessary to confirm the nursing impression. Chart review was also performed to assist in this determination. Demographics, admission diagnosis, vital signs and laboratory data that triggered the alert logic, and any treatment the associated clinical event required were also recorded. Chart review was repeated 48 hours later to review microbiological test results and physician progress notes that might shed further light on the clinical event that triggered the alert.

The “clinical event” associated with each alert was defined as the most likely acute explanation for the vital sign and laboratory abnormalities that triggered the alert. A clinical event might be an acute illness, such as pneumonia with septic shock, or a non-illness event, such as initiation of dialysis. Clinical events could include the illness that necessitated admission if the alert fired within 24 hours of admission, or secondary illnesses - for instance, a catheter-associated blood stream infection.

The severity of clinical events related to alert firings were classified into three tiers. 

1. Major clinical events were acute life-threatening illnesses that required emergent resuscitation with any one or more of the following: >1 L intravenous fluid resuscitation, vasopressor infusion, >2 units of packed red blood cell transfusion, endotracheal intubation, advanced cardiac life support, or emergent surgical intervention.
2. Minor clinical events were acute non-life-threatening illnesses that required urgent treatments not included in the definition of major clinical events above.
3. False alerts were said to have occurred when no acute illness was recognized in temporal relationship to the alert firing. 

The positive predictive value of the alert for detecting severe sepsis, major clinical events, and major or minor clinical events were calculated, with 95% confidence intervals.

Results

Nineteen women and 21 men, with ages ranging from 22 to 103 years were included. Twenty-two of forty (55%) were in the ICU at the time the alert fired, and 18 on the floors. Vital signs and laboratory values that triggered the alert logic are listed in Table 2. 

Table 2. SIRS / organ dysfunction alert trigger criteria in forty patients. 

Eleven of 40 patients (28%) had a major clinical event associated with the alert firing – two of these occurred outside the ICU. Severe sepsis or septic shock accounted for four of these major clinical events – yielding a positive predictive value of 0.10 (95%CI: 0.04-0.23) of the alert for detection of severe sepsis or septic shock. The seven remaining patients with major events suffered acute pulmonary edema, pulmonary embolism, ischemic bowel, pancreatitis, acute cardiogenic shock, acute right heart failure secondary to pulmonary hypertension, and an incarcerated enteric hernia. The positive predictive value of the alert for detection of major clinical events was 0.28 (95%CI: 0.16-0.43).

Major clinical events were clearly recognized before the alert fired in nine of 11 cases, as evidenced by the patient having been admitted or transferred to the intensive care unit specifically for the event of interest, and/or having received treatment such as intubation or initiation of intravenous vasopressors before the alert fired. In two cases, the alert fired at about the same time that treatment of the acute clinical event commenced, and it was unclear what role it played in clinical recognition of the event.

Seven of 40 patients (17%) had a minor clinical event associated with the alert firing. These included two patients with anemia, and one each with hypotension from an antihypertensive medication, dialysis disequilibrium, post-operative pain, dehydration, and paroxysmal atrial fibrillation. The positive predictive value of the alert for detecting either a major or minor clinical event was 0.45 (95%CI: 0.31-0.60).

Twenty-two of 40 patients (55%) were not experiencing any identifiable acute illness that could explain the alert firing - these were considered false alerts. Aberrant vital signs triggered false alerts during dialysis (2), turning or sitting-up post-operative patients (2), an endoscopy procedure, and a family argument. Other false alerts were attributable to the pharmacological effect of calcium channel blocker, oximeter malfunction, error in vital sign entry, and widely discrepant blood pressures between right and left arms. The remaining false alerts were triggered by slightly abnormal vital signs with no identifiable cause.

Four patients (10%) did not survive to discharge – two had major clinical events, one a minor clinical event and one a false alert – in the later two cases, the cause of death was unrelated to the clinical event that triggered the alert.  

We examined alert triggering criteria to better understand how the discriminant ability of the alert might be improved. We noted that 15 of 40 (37%) alerts triggered with respiratory rates of 21 or 22 bpm, however these included six alerts associated with major clinical events. Twelve of 40 (30%) alerts triggered with heart rates in 91-95 bpm range, including two alerts associated with major clinical events. Laboratory results contributed to 31 of 40 alert firings – but in 12 cases they were stable or improving at the time they triggered the alert. In no case was a stable or improving laboratory value associated with a major clinical event.

Discussion

It’s important to study the effects of any quality improvement project in order to determine whether it is having the desired results. Our small pilot study suggests that our first-generation SIRS/organ dysfunction alert has a low positive predictive value for severe sepsis, and generates many false alerts. This is partially a reflection of the low specificity of SIRS criteria for sepsis (6). The high number of false positive alerts has led to alert-fatigue among physicians and nurses providing bedside patient care – a phenomenon which is not unique to our institution (7).  

Our alert demonstrated greater potential utility to detect acute clinical deterioration than to detect sepsis. Buck and colleagues (7) used an EMR-based logic system to activate a sepsis alert similar to ours, and observed similar results in that only 17% of alert patients had a sepsis-related discharge diagnosis, but 40% had a major illness which required urgent intervention. We have used the results of our study to re-task future iterations of our alert to detect acute clinical deterioration rather than sepsis.

Other researchers provide guidance in this regard. Vital sign and laboratory result criteria similar to the ones used in our study have been previously shown to predict in-hospital cardiac arrest (8), predict 30-day mortality (9), generate early warning scores to detect acute clinical deterioration (9), and activate medical emergency teams (8,10). A recent large study by Churpek and colleagues (11) validated a risk stratification tool that utilized vital signs, laboratory findings and demographics to predict the combined outcome of cardiac arrest, ICU transfer or death on the wards. The model yielded notable discriminant accuracy with an area under the receiver operating curve (AUROC) of 0.77.

We are currently investigating revisions in our alert logic to improve detection of acute clinical deterioration. The alert logic now trends laboratory values associated with organ dysfunction. We are studying whether adding a reflex serum lactate to the automatic alert response might help identify patients who are acutely deteriorating (12).   

Our study has many apparent weaknesses, but it should be noted that it was carried out originally only to provide data to help guide local efforts to improve patient safety. In this regard, it succeeded in guiding our (and perhaps other’s) future efforts in what will more likely be a useful direction.

We failed to clearly determine what role our automated alert played in bedside decision-making. In most cases, clinicians were already evaluating or treating the clinical event that triggered the alert before the alert fired. However, we feel that a safety net is a wise precaution even in a high-reliability system. It should also be noted that our institution has medicine and surgery residency teaching programs, a critical care fellowship, 24/7 in-house intensivist coverage, and remote video ICU coverage. The benefit of EMR-based automated alerts is likely to be amplified in less well-staffed institutions. Refined versions of EMR-based automated alerts, such as the ones we are currently investigating, have potential to enhance patient safety. 

References

1. Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303-10. [CrossRef] [PubMed]
2. Rivers E, Nguyen B, Havstad S, Ressler J, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345:1368-77. [CrossRef] [PubMed]
3. The ProCESS investigators. A randomized controlled trial of protocol-based care for early septic shock. N Engl J Med. 2014;370(18):1683-93. [CrossRef] [PubMed]
4. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, Cohen J, Opal SM, Vincent JL, Ramsay G. International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250-6. [CrossRef] [PubMed]
5. Dellinger RP, Levy MM, Rhodes A, et al, Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580. [CrossRef] [PubMed]
6. Pittet D, Range-Frausto S, Tarara LN, et al. Systemic inflammatory response syndrome, sepsis, severe sepsis and septic shock: incidence, morbidities and outcomes in surgical ICU patients. Intensive Care Med. 1995;21:302-9. [CrossRef] [PubMed]
7. Buck KM. Developing an early sepsis alert program. J Nurs Care Qual. 2014;29(2):124-32. [CrossRef] [PubMed]
8. Hodgetts TJ, Kenward G, Ioannis G, et al. The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team. Resuscitation. 2002;54:125-31. [CrossRef] [PubMed]
9. Goldhill DR, McNarry AF. Physiological abnormalities in early warning scores are related to mortality in adult inpatients. Br J Anaesth. 2004;92:882-4. [CrossRef] [PubMed]
10. Kenward G, Castle N, Hodgetts T, Shaikh L. Evaluation of a medical emergency team one year after implementation. Resuscitation. 2004;61:257-63. [CrossRef] [PubMed]
11. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development of validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190:649-55. [CrossRef] [PubMed]
12. Bakker J, Nijsten MWN, Jansen TC. Clinical use of the lactate monitoring in critically-ill patients. Ann Intensive Care. 2013;3:12-20. [CrossRef] [PubMed] 

Reference as: Raschke RA, Owen-Reece H, Khurana H, Groves RH Jr, Curry SC, Martin M, Stoffer B, Uppalapu S, Seth H, Menon N. Clinical performance of an automated systemic inflammatory response syndrome (sirs) / organ dysfunction alert: a system-based patient safety project. Southwest J Pulm Crit Care. 2014;9(4):223-9. doi: http://dx.doi.org/10.13175/swjpcc121-14 PDF