An Observational Study Demonstrating the Efficacy of Interleukin-1 Antagonist (Anakinra) in Critically-ill Patients with Hemophagocytic Lymphohistiocytosis

Kyle Henry MD, Banner University Medical Center

Robert Raschke MD, University of Arizona College of Medicine-Phoenix

Phoenix, AZ USA

Abstract

Secondary Hhmophagocytic lymphohistiocytosis (HLH) is an underrecognized cause of multisystem organ failure (MSOF) in critically ill adults, associated with high mortality even when recommended etoposide-based treatments are administered.  Anakinra, an interleukin-1 receptor antagonist, has shown promise in treating children with HLH. This retrospective case series describes seven adult patients who presented to our ICU with a unremitting syndrome consistent with sepsis / MSOF, who were subsequently diagnosed with secondary HLH and received anakinra.   Five of seven (71%) survived.  Two non-survivors died secondary to opportunistic fungal infections. Our study contributes to mounting observational evidence regarding anakinra’s possible efficacy in critically ill adults with HLH, and also raises awareness of possible infectious complications of its use. 

Introduction

Hemophagocytic lymphohistiocytosis (HLH) is a syndrome characterized by immune dysregulation, hypercytokinemia and tissue infiltration by activated cytotoxic lymphocytes and macrophages (1-3). Primary HLH is a familial syndrome in which gene mutations causing abnormalities of cytotoxic T-lymphocyte and natural killer (NK) cell function result in a systemic hyperinflammatory state. Primary HLH typically presents in the first years of life, progressing to multisystem organ failure (MSOF) and death unless successfully treated with chemotherapy and bone marrow transplantation. Secondary HLH shares clinical features with primary HLH but typically occurs later in life after an underlying illness triggers a dysregulated inflammatory response (3, 4). The diagnosis of primary or secondary HLH is made when five of eight criteria proposed by the International Histiocyte Society are met (Table 1) (5). Heterogeneous groups of patients may satisfy HLH diagnostic criteria, including those in whom HLH is triggered by sepsis, malignancy, and rheumatologic disease (4). Macrophage Activation Syndrome (MAS) is a specific HLH subcategory describing those patients with secondary HLH due to underlying rheumatologic disease (2). The clinical course of secondary HLH is highly variable, progressing relatively slowly in some patients in whom a diagnosis may be made in an outpatient oncology or rheumatology clinic (2, 4, 6-9). Other patients deteriorate rapidly and may require ICU care before the diagnosis of HLH is suspected (3). It has been increasingly recognized that subgroups of patients with HLH have distinctive clinical features, and require special treatment considerations (1, 3-5, 10).

One distinct subgroup consists of adults who present to the intensive care unit (ICU) with sepsis syndrome and MSOF with progressive deterioration despite standard therapy for sepsis (3). Life threatening manifestations in such patients suspected of experiencing HLH may force consideration of presumptive immunotherapy before all HLH diagnostic tests have resulted. Infectious and/or rheumatologic triggers for secondary HLH are eventually found in many, but a clear distinction between sepsis and HLH cannot be made in some (3, 4, 10, 11). The standard treatment protocol for HLH incorporates etoposide – a myelosuppressive chemotherapy agent generally regarded as the standard of care, but has known serious side effects, especially in the setting of hepatic or renal dysfunction typical of sepsis (5, 12). Furthermore, etoposide-based HLH treatment may cause severe immunosuppression leading to opportunistic infections. The mortality of secondary HLH in the adult ICU exceeds 50% (13-16) regardless of the underlying catalyst for the hypercytokinemia. New therapeutic options are desperately needed.

Mounting observational evidence suggests that Anakinra, a recombinant interleukin-1 receptor antagonist (IL-1Ra), may have promise in the treatment of HLH (6,17). Naturally-occurring IL-1Ra is secreted by immune cells to inhibit the pro-inflammatory effects of interleukin 1β (IL-1β) – a key cytokine in the pathogenesis of sepsis and HLH (7, 18). Anakinra was originally developed as a potential therapy for sepsis (7), but is now FDA-approved for use in rheumatoid arthritis. A single case-series describes the successful use of anakinra in critically-ill children with secondary HLH and a few case reports describe its use in critically-ill adults (6, 8, 19, 20). More recently, Wohlfarth and colleagues showed that anakinra is a reasonable option for critically ill patients adults with HLH.  At the same time Wohlfarth et al were studying these effects in an Austrian population, we demonstrated similar results in a series of adult patients in the United States who presented to the ICU with sepsis syndrome and underwent treatment with anakinra for secondary HLH.

Methods

This retrospective study was approved by our institutional review board. The setting was the medical and surgical ICU at Banner-University Medical Center Phoenix – a 72-bed ICU in a 650-bed academic tertiary referral center. We identified consecutive adult patients at least 18 years old admitted with sepsis syndrome (known or suspected infection plus acute organ system dysfunction) (21) who subsequently met five or more HLH-2004 diagnostic criteria (Table 1) and received anakinra as part of their treatment regimen between May 2013 and May 2016.

Table 1. HLH-2004 Diagnostic Criteria for Secondary HLH: At least five of eight criteria needed for diagnosis.

Clinical management of patients was not strictly protocolized, but care was provided by an academic 24/7 on-site intensivist service with strong internal consensus regarding the management of HLH. All patients had at least daily complete blood counts and basic metabolic panels. In our practice, diagnostic workup for HLH generally commences upon recognition of unremitting sepsis syndrome with MSOF and bicytopenia. Such patients underwent workup for sepsis and potential causes of secondary HLH that included at minimum: blood cultures, ferritin, fibrinogen, triglycerides, bone marrow aspiration and biopsy, PCR and/or serological testing for systemic lupus erythematosus (SLE), Epstein-Barr virus (EBV), cytomegalovirus, herpes simplex virus, human immunodeficiency virus, hepatitis viruses and coccidioidomycosis (a mycosis endemic in the region). The decision to start HLH therapy was typically based on clinical suspicion plus consistent preliminary laboratory results such as hyperferritinemia, hypofibrinogenemia and/or hypertriglyceridemia, while awaiting the complete results of bone marrow aspiration/biopsy and send-out tests such as soluble interleukin-1 receptor and NK cell functional assays. Presumptive treatment of HLH began with corticosteroids - typically intravenous dexamethasone 10mg/m² daily. Additional therapies were added at the discretion of the intensivist with consideration of the rapidity of clinical deterioration and likely intolerance of some therapies due to kidney, liver, and/or bone marrow failure. Choice of HLH therapies was based on the HLH-1994 therapy protocol, and influenced by our prior unfavorable experience with etoposide (discussed in conclusions) and recognition of observational literature suggesting that anakinra might be efficacious in patients with secondary HLH. Anakinra was typically given in a dose of 100mg subcutaneously daily, except in patients with creatinine clearance <30ml/min who were dosed every other day.

We retrospectively performed chart reviews to abstract demographics and clinical features related to sepsis and MSOF including infections present on admission, mental status, acute respiratory failure requiring mechanical ventilation, acute renal failure requiring hemodialysis, shock requiring intravenous vasopressors, and liver injury (defined as total bilirubin >2 mg/dL and aminotransferase greater than two times upper limit of normal) (22). The sequential organ failure assessment (SOFA) score was calculated for each patient (21). HLH-2004 diagnostic criteria and the underlying disease process thought to have triggered HLH were abstracted. We documented all treatments including antibiotics and immunosuppressive therapy for HLH, including the dose and duration of anakinra. Outcomes included survival to hospital discharge, duration of fever, mechanical ventilation and renal replacement therapy and ICU length of stay indexed to the time anakinra commenced. Infectious complications occurring during admission after HLH therapy started were also documented. Simple descriptive statistics were performed.

The H Score for each patient was also calculated retrospectively. The H Score is a score used to estimate an individual's risk for having secondary HLH and was recently validated in a 147 patient cohort by Debaugnies et al. (23).

Results

Seven patients were treated with anakinra for a diagnosis of secondary HLH in our ICU between May 2013 and May 2016. Patient ages ranged from 22-59 years – three were female. All patients initially presented to our ICU with a febrile illness consistent with sepsis and received broad-spectrum intravenous antibiotics. Microbiological testing eventually documented infections in two patients – due to influenza A and EBV, respectively. All patients were encephalopathic, five required mechanical ventilation, four required hemodialysis due to acute renal failure, four had liver injury and three required vasopressors due to shock (Table 2).

Table 2. Patient characteristics and some clinical outcomes.

The median SOFA score was 13 (range: 3-17) predicted poor outcome for the group overall (13-16). H Scores for this cohort ranged from 122-263.  HLH diagnostic criteria and presumed etiologies are listed in Table 3.

Table 3. Suspected etiology and positive HLH-2004 diagnostic criteria for secondary HLH in patients treated with anakinra: Five of eight criteria required to diagnose HLH.

Two patients were known to have SLE prior to ICU admission and four others were subsequently diagnosed with underlying autoimmune diseases demonstrating a preponderance of MAS in this cohort.  

All patients initially received corticosteroids (dexamethasone 10mg/m² or methylprednisolone >500mg every 12 hours) followed by anakinra. Three patients also received cyclosporine, three underwent plasmapheresis and two received IVIG. Only one patient received etoposide and this was later transitioned to anakinra due to lack of response. Anakinra was started a median of seven days after ICU admission (range 2-58 days). All patients received anakinra 100mg daily, but Q48 hour dosing was used temporarily in five patients who transiently experienced creatinine clearances <30mL/min. Duration of anakinra therapy was 10-159 days - we were unable to determine duration of anakinra after discharge in one patient.

All patients appeared to clinically improve after initiation of anakinra. Of six patients experiencing fever at the time anakinra was started, five defervesced within 24 hours. In five patients that had follow-up ferritin levels within two weeks of starting anakinra, ferritin fell from a median of 7,371 ng/L (range 2,217->40,000) to 4,535 ng/L (range 2,137-26,634). Five of seven patients (71%) survived to hospital discharge with an ICU length of stay (LOS) ranging from 6-17 days and an overall LOS of 17-103 days. Once anakinra was started, liberation from the ventilator occurred within 1-3 days, transfer out of the ICU within 3-5 days, discharge from the hospital within 10-32 days and discontinuation of hemodialysis within 10-44 days.

Death in both non-survivors was due to opportunistic fungal infections - necrotizing pulmonary aspergillosis and disseminated mucormycosis (which occurred despite prophylaxis with amphotericin B).  Three other secondary infections all occurred in a single survivor: methicillin-sensitive S. aureus and E coli bacterial ventilator-associated pneumonias and C. difficile colitis all of which responded favorably to treatment while anakinra was continued.

Discussion

Secondary HLH may be more common in the ICU than previously recognized (3), overlapping with and at times indistinguishable from sepsis (3, 4, 10, 11). Rapid clinical deterioration in patients with suspected HLH may force treatment decisions to be made before full diagnostic test results are available. The risk of myelosuppression due to etoposide-based HLH treatment regimens may be intolerable in critically-ill, possibly septic patients with MSOF (4, 12). A therapeutic agent with a more HLH-specific mechanism of action and better safety profile is badly needed.

Anakinra is a recombinant IL-1Ra originally investigated as a potential immune-modulatory treatment for sepsis (7). Phase I and II studies established acceptable safety for further study in sepsis, but a phase III trial failed to demonstrate an overall survival benefit (7). A post-hoc analysis of data from this trial showed that septic patients with hepatobiliary dysfunction, hypofibrinogenemia and thrombocytopenia, such as often seen in secondary HLH, had significantly improved survival if they received anakinra vs placebo (65% vs. 35% 28-day survival, p=0.0007) (7). Anakinra was later approved for use in rheumatoid arthritis (18). Observational studies suggested efficacy in adult onset Still’s disease and systemic juvenile arthritis (9, 24, 25) and in non-critically-ill patients with secondary HLH triggered by these rheumatologic diseases (9, 25, 26). Case reports described the use of anakinra in critically-ill children with secondary HLH (25, 26, 27) and Rajasekaran and colleagues published a case series describing their experience using anakinra in eight critically-ill pediatric patients with secondary HLH/sepsis syndrome (6). All eight patients survived their initial illness, and no infectious complications were attributed to anakinra.

Fourteen cases describing the use of anakinra in adults with secondary HLH have previously been published (6, 8 ,17, 19, 20, 28).  Four were due to infections (EBV, CMV, MAC, histoplasmosis, four to autoimmune disease (two with AOSD, SLE antisynthetase syndrome), two post transplantation immunosuppression, one due to acute lymphocytic leukemia and three of unknown trigger.  Twelve of 15 (80%) required life support (mechanical ventilation, hemodialysis, vasopressors).  All but two received corticosteroids and just over half IVIg.  Overall survival was 67%, and no complications of immunosuppression were reported. 

The mechanism by which anakinra might ameliorate secondary HLH is not fully elucidated. Secondary HLH (and some forms of sepsis) are characterized by high levels of circulating cytokines including interleukin-6 (IL-6), tumor necrosis factor and interferon-gamma (IFN-γ) (2, 4, 7, 18, 29) - constituting what some have called a “cytokine storm”. Many investigators believe that hypercytokinemia is pathogenic in HLH. Renal failure, cytopenia, coagulopathy and cholestasis have been associated with elevated levels of IL-6 and IFN-γ (2, 18, 28). IL-1β activates lymphocytes responsible for production of these same cytokines (2, 7, 18, 28). IL-1Ra is a competitive inhibitor of IL-1β (7, 29). Therefore IL-1 receptor antagonism by anakinra might inhibit the maladaptive hypercytokinemia characteristic of secondary HLH. This brief explanation oversimplifies a complex and poorly-understood process that requires much further research.

The observed survival rate in our adult patients treated with anakinra (71%) appears favorable compared to that described in other comparable groups of patients (13-16) with survival rates ranging from 25-41%, although we cannot definitively attribute this to treatment effect. It is notable that the majority of our patients had MAS, which has a improved prognosis compared to other forms of HLH when it presents in the outpatient setting, but similar high mortality once the patient develops MSOF and requires intensive care (13-16). This finding supports the concept that secondary HLH of any cause is related to a cytokine storm universal to all underlying catalysts, and that after a critical point the inflammatory cascade becomes increasingly difficult to reverse.

We have previously diagnosed and treated a total of 29 cases of secondary HLH in our ICU. Survival among 22 patients who did not receive anakinra was 14%. This group included eight patients who received etoposide, all of whom died or developed severe neutropenia (WBC <0.5 X 10⁹/L) within a week of its initiation. The patient who survived etoposide did so after her regimen transitioned to anakinra. Statistical comparison of patients in our practice who did or did not receive anakinra was not undertaken due to potential bias and confounding. Controlled prospective trials are required to determine whether anakinra will improve survival of patients with secondary HLH. One such trial is currently under way (ClinicalTrials.gov Identifier: NCT02780583) specifically in regards to MAS.

Two of our patients died from opportunistic fungal infections. Fatal fungal infections have previously been reported to occur in patients receiving treatment for HLH who did not receive anakinra (28) and are likely a result of multiple risk factors including the underlying immune dysregulation associated with HLH, other immunosuppressive therapies and invasive procedures related to ICU care (30, 31), and therefore these infections cannot be specifically attributed to anakinra. We consider prophylactic posaconazole or amphotericin therapy in selected ICU patients at high risk for fungal infections given the evidence of invasive fungal infection prophylaxis including mucormycosis in similarly immunocompromised patients (30, 31). 

Conclusions

Our study contributes to mounting observational evidence supporting the hypothesis that anakinra may be efficacious in adult patients presenting to the ICU with life-threatening secondary HLH.  In our opinion, it can be considered as first line therapy, in combination with corticosteroids and IVIg, in selected patients for whom renal, hepatic and bone marrow dysfunction put them at higher risk of toxicity due to etoposide.  Vigilance is warranted in relation to opportunistic infections, particularly those due to fungi.  Prospective controlled trails are needed to definitively establish effective therapy of HLH. 

References

1. Janka GE, Lehmberg K. Hemophagocytic lymphohistiocytosis:pathogenesis and treatment. Hematology: American Society of Hematology Educational Program 2013:605-11. [CrossRef] [PubMed]
2. Schulert GS, Grom AG. Pathogenesis of macrophage activation syndrome and potential for cytokine-directed therapies. Annu Rev Med. 2015:66;145-59. [CrossRef] [PubMed]
3. Raschke RA, Garcia-Orr R. Hemophagocytic lymphohistiocytosis:A potentially underrecognized association with systemic inflammatory response syndrome, severe sepsis, and septic shock in adults. Chest. 2011;140:933-8. [CrossRef] [PubMed]
4. Castillo L, Carcillo J. Secondary hemophagocytic lymphocytosis and severe sepsis/systemic inflammatory response syndrome/multiorgan dysfunction syndrome/macrophage activation syndrome share common intermediate phenotypes on a spectrum of inflammation. Pediatr Crit Care Med. 2009;10:387-92. [CrossRef] [PubMed]
5. Henter J, Horne A, Aricó M, et al. HLH-2004:Diagnostic and therapeutic guidelines for hemophagocytic lymphohistiocytosis. Pediatr Blood Cancer. 2007;48:124-31. [CrossRef] [PubMed]
6. Rajesekaran S, Kruse K, Kovey K, Davis AT, Hassan NE, Ndika AN, Zuiderveen S, Birmingham J. Therapeutic role of anakinra, an interleukin-1 receptor antagonist, in the management of secondary hemophagocytic lymphohistiocytosis/sepsis/multiple organ dysfunction/macrophage activating syndrome in critically ill children. Pediatr Crit Care Med. 2014;15:401-8. [CrossRef] [PubMed]
7. Shakoory B, Carcillo JA, Chatham WW, Amdur RL, Zhao H, Dinarello CA, Cron RQ, Opal SM. Interleukin-1 receptor blockade is associated with reduced mortality in sepsis patients with features of macrophage activation syndrome:reanalysis of a prior phase III trial. Crit Care Med. 2016;44:275-81. [CrossRef] [PubMed]
8. Loh NK, Lucas M, Fernandez S, Prentice D. Successful treatment of macrophage activation syndrome complicating adult Still's disease with anakinra. Intern Med J. 2012;42:1358-62. [CrossRef] [PubMed]
9. Lenert A, Yao Q. Macrophage activation syndrome complicating adult onset Still's disease:a single center case series and comparison with literature. Semin Arthritis Rheum. 2016;45:711-6. [CrossRef] [PubMed]
10. Tothova Z, Berliner N. Hemophagocytic syndrome and critical illness: new insights into diagnosis and management. J Intensive Care Medicine. 2014;30:401-12. [CrossRef] [PubMed]
11. Stéphan F, Thiolière B, Verdy E, Tulliez M. Role of hemophagocytic histiocytosis in the etiology of thrombocytopenia in patients with sepsis syndrome or septic shock. Clin Inf Dis. 1997;25:1159-64. [CrossRef] [PubMed]
12. Donelli MG, Zucchetti M, Munzone E, D'Incalci M, Crosignani A. Pharmacokinetics of anticancer agents in patients with impaired liver function. Eur J Cancer. 1998;34:33-46. [CrossRef] [PubMed]
13. Park, HS, Kim DY, Lee JH, et al. Clinical features of adult patients with secondary hemophagocytic lymphohistiocytosis from causes other than lymphoma:an analysis of treatment outcome and prognostic factors. Ann Hematol. 2012;91:897-904. [CrossRef] [PubMed]
14. Kaito K, Kobayashi M, Katayama T, et al. Prognostic factors of hemophagocytic syndrome in adults:analysis of 34 cases. Eur J Haematol. 1997;59:247-53. [CrossRef] [PubMed]
15. Li J, Wang Q, Zheng W, Ma J, Zhang W, Wang W, Tian X. Hemophagocytic lymphohistiocytosis:clinical analysis of 103 adult patients. Medicine. 2014;93:100-5. [CrossRef] [PubMed]
16. Shabbir M, Lucas J, Lazarchick J, Shirai K. Secondary hemophagocytic syndrome in adults:a case series of 18 patients in a single institution and a review of literature. Hematol Oncol. 2011;29:100-6. [CrossRef] [PubMed]
17. Wohlfarth P, Agis H, Gualdoni GA, et al. Interleukin I receptor antagonist anakinra, intervenous immunoglobulin, and corticosteroids in the management of critically ill adult patients with hemophagocytic lymphohistiocytosis. J Intensive Care Med. 2017 Jan 1:885066617711386. [CrossRef] [PubMed]
18. Gabay C, Lamacchia C, Palmer G. IL-1 pathways in inflammation and human diseases. Nat Rev Rheumatol. 2010;6:232-41. [CrossRef] [PubMed]
19. Mehta MV, Manson DK, Horne EM, Haythe J. An atypical presentation of adult-onset Still's disease complicated by pulmonary hypertension and macrophage activation syndrome treated with immunosuppression:a case-based review of the literature. Pulm Circ. 2016;6:136-42. [CrossRef] [PubMed]
20. Divithotawela C, Garrett P, Westall G, Bhaskar B, Tol M, Chambers DC. Successful treatment of cytomegalovirus associated hemophagocytic lymphohistiocytosis with the interleukin 1 inhibitor anakinra. Respir Case Rep. 2016;4:4-6. [CrossRef] [PubMed]
21. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis:for the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315:762-74. [CrossRef] [PubMed]
22. Sands KE, Bates DW, Lanken PN, et al. Epidemiology of sepsis syndrome in 8 academic medical centers. JAMA. 1997;278:234-40. [CrossRef] [PubMed]
23. Debaugnies F, Mahadeb B, Ferster A, Meuleman N, Rozen L, Demulder A, Corazza F. Performances of the H-Score for diagnosis of hemophagcytic lymphohistiocytosis in adult and pediatric patients. Am J Clin Pathol. 2016;145:862-70.[CrossRef] [PubMed]
24. Nigrovic PA, Mannion M, Prince FH, et al. Anakinra as first-line disease-modifying therapy in systemic juvenile idiopathic arthritis:report of forty-six patients from an international multicenter series. Arthritis Rheum. 2011;63:545-55. [CrossRef] [PubMed]
25. Bruck N, Suttorp M, Kabus M, Heubner G, Gahr M, Pessler F. Rapid and sustained remission of systemic juvenile idiopathic arthritis-associated macrophage activation syndrome through treatment with anakinra and corticosteroids. J Clin Rheumatol. 2011;17:23-27. [CrossRef] [PubMed]
26. Miettunen PM, Narendran A, Jayanthan A, Behrens EM, Cron RQ. Successful treatment of severe paediatric rheumatic disease-associated macrophage activation syndrome with interleukin-1 inhibition following conventional immunosuppressive therapy: case series with 12 patients. Rheumatol. 2011;50:417-9. [CrossRef] [PubMed]
27. Kelly A, Ramanan AV. A case of macrophage activation syndrome successfully treated with anakinra. Nat Clin Pract Rheumatol. 2008;4:615-20. [CrossRef] [PubMed]
28. Ocon, AJ, Bhatt BD, Miller C and Peredo RA. Safe usage of anakinra and dexamethasone to treat refractory hemophagocytic lymphohistiocytosis secondary to acute disseminated histoplasmosis in a patient with HIV/AIDS. BMJ Case Rep. 2017 Oct 4;2017. pii: bcr-2017-221264. [CrossRef] [PubMed]
29. Dinarello CA. Interleukin-1 and its biologically related cytokines. Adv Immunol. 1989;44:153-205. [CrossRef] [PubMed]
30. Lawrence TM, Thabet A, Nishino H. Case 10-2011 - a woman with fever, confusion, liver failure, anemia, and thrombocytopenia. N Engl J Med. 2011;364:1259-70. [CrossRef] [PubMed]
31. Bajwa SJ, Kulshrestha A. Fungal infections in intensive care unit:challenges in diagnosis and management. Ann Med Health Sci Res. 2013;3:238-44. [CrossRef] [PubMed]
32. Cornely OA, Maertens J, Winston DJ, et al. Posaconazole vs. fluconazole or itraconazole prophylaxis in patients with neutropenia. N Engl J Med. 2007;356:348-59. [CrossRef] [PubMed]

Cite as: Henry K, Raschke RA. An observational study demonstrating the efficacy of interleukin-1 antagonist (anakinra) in critically-ill patients with hemophagocytic lymphohistiocytosis. Southwest J Pulm Crit Care. 2019;18(6):177-86. doi: https://doi.org/10.13175/swjpcc034-19 PDF 

Editor's Note: The July 2019 Critical Care Case of the Month is a case presentation of HLH with 0.5 Hour CME credit. Click on the link to be directed to the case.

Clinical Performance of an Interactive Clinical Decision Support System for Assessment of Plasma Lactate in Hospitalized Patients with Organ Dysfunction

Robert A. Raschke, MD MS

Hargobind Khurana, MD 

Huw Owen-Reece, MBBS 

Robert H. Groves Jr, MD

Steven C. Curry, MD

Mary Martin, PharmD

Brenda Stoffer, RN BSN

Banner University Medical Center Phoenix

Phoenix, AZ USA

Abstract

Purpose: Elevated plasma lactate concentration can be a useful measure of tissue hypo-perfusion in acutely deteriorating patients, focusing attention on the need for urgent resuscitation. But lactate is not always assessed in a timely fashion in patients who have deteriorating vital signs. We hypothesized that an electronic medical record (EMR)-based decision support system could interact with clinicians to prompt assessment of plasma lactate in appropriate clinical situations in order to risk stratify a population of inpatients and identify those who are acutely deteriorating in real-time.

Methods: All adult patients admitted to our hospital over a three month period were monitored by an EMR-based lactate decision support system (lactate DSS) programmed to detect patients exhibiting acute organ dysfunction and engage the clinician in the decision to order a plasma lactate concentration. Inpatient mortality was determined for the five risk categories that this system generated, and chart review was performed on a high-risk subgroup to describe the spectrum of bedside events that triggered the system logic.

Results: The lactate DSS segregated inpatients into five strata with mortality rates of 0.8% (95%CI:0.6-1.0%); 2.7% (95%CI:1.0-4.4%); 7.9% (95%CI: 6.0-10.1%), 13.0% (95%CI: 9.0-17.8%) and 42.1% (95%CI: 32.0-52.4%), achieving a discriminant accuracy of 80% (95%CI:76-84%) by AUROC for predicting inpatient mortality. Classification into the two highest risk strata had a positive predictive value for detecting acute life-threatening clinical events of 54% (95%CI: 41.5-66.5%).

Conclusions: Our lactate decision support system is different than previously-described computerized “early warning systems”, because it engages the clinician in decision-making and incorporates clinical judgment in risk stratification. Our system has favorable operating characteristics for the prediction of inpatient mortality and real-time detection of acute life-threatening deterioration.

Introduction

Over 700,000 deaths occur annually in U.S. hospitals (1). Sepsis accounts directly for 37% and indirectly for 56% of these deaths (2). Other common causes of inpatient mortality such as acute hemorrhage and venous thromboembolism (3) share certain early clinical findings with sepsis, in that they may present with deterioration of vital signs and biochemical variables before life-threatening manifestations become obvious (4). Recognition of these findings provides an opportunity for early intervention, which has been shown to improve mortality (5,6). Studies have shown that failure to rapidly recognize acute clinical deterioration is one of the most common root causes of preventable inpatient mortality (4,8).

Early warning systems (EWSs) are a type of clinical decision support system (CDSS) utilized to provide surveillance of hospitalized patients in order to alert clinicians when a patient has findings associated with acute deterioration (19). These typically monitor for abnormal vital signs or laboratory evidence of organ dysfunction, but have included many other types of clinical and laboratory variables (20-23). Modern EWSs utilize logistic regression to weight up to 36 different independent variables and yield highly stratified risk scores (24-26).

We had previous experience developing a simple EWS that triggered when at least two systemic inflammatory response syndrome (SIRS) criteria plus at least one of 14 acute organ dysfunction (OD) parameters was detected. Although this system references SIRS it was found to be nonspecific for sepsis (27), and was subsequently employed in our healthcare system to identify patients deteriorating in real-time regardless of the cause. Subsequent research showed that our SIRS/OD alert system was triggered during the course of 19% of admissions, and that patients who triggered the alert had an odds ratio of 30.1 (95% CI: 26.1-34.5) for inpatient mortality (28). We hypothesized that this SIRS/OD alert system could be used to identify high risk patients who might be further risk-stratified by obtaining a plasma lactate concentration.

Elevated plasma lactate concentration is a particularly useful biochemical marker of acute decompensation. Hyperlactemia is pathophysiologically associated with acute tissue hypoperfusion, and clinically associated with organ dysfunction and mortality (7-11). Hyperlactemia is also associated with the need for urgent clinical interventions such as transfusion and urgent surgery in trauma patients (13,14), and resuscitation of medical patients with sepsis or other life-threatening illnesses (5,15). Lactate assessment is integral to the definition of sepsis (7,16), and an essential component of the Surviving Sepsis Campaign sepsis resuscitation bundle (6). Lactate assessment is integral to achieving sepsis bundle compliance as defined by the Centers for Medicare and Medicaid Services (CMS), which has mandated participating hospitals to report as a measure of quality of care. However, lactate is only ordered about half the time that it ought to be in patients with severe sepsis and septic shock (17,18). To our knowledge, only one previously reported EWS incorporates lactate assessment (29), but this system passively utilized lactate concentration results obtained on admission from the emergency room and was not used for surveillance during hospitalization.

We sought to use our SIRS/OD alert system to actively trigger lactate assessment to identify patients suffering from sepsis or any other life-threatening disease process requiring immediate intervention during hospitalization. We hypothesized that the resulting “lactate decision support system” (lactate DSS) would provide inpatient mortality risk stratification with high discriminant accuracy, and detect acute life-threatening events with high positive predictive value compared to contemporary EWSs.

A lactate DSS with these favorable characteristics could theoretically be used to guide emergent interventions in an effort to save lives, although it was not our aim at this time to perform an interventional trial. The specific aims of this study were to pilot an interactive lactate DSS in our healthcare system, and to calculate its discriminant accuracy for mortality risk stratification, and its positive predictive value as a real-time early warning system.

Methods

We prospectively studied a cohort of all adult inpatients admitted to Banner-University Medical Center - Phoenix, a 650-bed academic hospital in Phoenix Arizona, during the first quarter of 2014. Our research was part of an ongoing system-level patient safety project and was approved by our Institutional Review Board.

The decision support logic was developed at Banner Health using Discern Expert^® (Cerner Corporation, North Kansas City MO, USA). The lactate decision support system (lactate DSS) monitored each patient in our EMR for vital signs and laboratory results consistent with SIRS and organ dysfunction, using criteria derived from the standard definition of sepsis (5-7) (Table 1).

Table 1. Lactate DSS trigger logic

If criteria for SIRS and organ dysfunction overlapped in any eight-hour window, the lactate DSS was triggered to respond. An electronic notification was generated to the patient’s nurse and physician alerting them to the possibility of acute clinical deterioration suggested by SIRS and organ dysfunction, and recommending evaluation and resuscitation if appropriate. Decision support included automatic generation of an order for a STAT plasma lactate if one was not previously ordered by the clinician, interactively prompting the clinician to cancel it if they felt it was unnecessary.

Adult admissions during the three-month study period and subsequent inpatient mortality were enumerated using our hospital’s general financial database: MedSeries4^® (Siemens Corporation, Washington DC). Although some patients triggered the lactate DSS multiple times over the course of their hospital stay, only the first trigger event was included in our analysis.

Inpatient mortality rates with ninety-five percent confidence intervals were calculated for each of five subgroups: 1) patients who did not exhibit SIRS and organ dysfunction during their hospitalization and therefore did not trigger a lactate DSS response; 2) patients who triggered a lactate DSS response, for whom a DSS-generated lactate order was cancelled by their clinician; 3) patients who triggered the lactate DSS and had a lactate concentration <2.2 mmol/L (normal for our laboratory); 4) patients who triggered the lactate DSS and had an elevated lactate of 2.2-3.9 mmol/L; and 5) patients who triggered the lactate DSS and had a highly elevated lactate >4.0 mmol/L.

It was our hypothesis that mortality in patients who triggered the lactate DSS logic would be equivalent whether the clinician chose not to cancel a DSS-generated lactate order, or the clinician had already entered a lactate order themselves. Therefore, we classified patients into the subgroups above regardless of whether their lactate order was DSS-generated or entered independently by the clinician. In order to confirm the validity of this hypothesis, the mortality rate of all patients with any lactate concentration result (the sum of groups 3, 4 and 5 above), and mortality rates within each lactate concentration strata, were separately analyzed to determine if mortality depended on the method of lactate order entry.

Stratified likelihood ratios and the area under the receiver operating curve (AUROC) generated using the five subgroups described above were calculated to determine the discriminant accuracy of the lactate DSS for the outcome of inpatient mortality.

A subgroup analysis was performed of all study patients with an elevated lactate >2.2 mmol/L (above the upper limit of normal range at our laboratory) detected by a DSS-generated lactate order during the first six weeks of the study. These patients’ charts were reviewed in order to characterize the acute clinical events that triggered a lactate DSS response in this subgroup of patients. A physician researcher reviewed progress notes, laboratory and microbiology results at the time of system activation, and for 72 hours afterwards to make this determination. Patient were determined to be suffering an acute life-threatening clinical event if a new-onset or rapidly-progressive disease process was present at the time the lactate DSS was triggered that required emergent treatment with any one of the following: >1 L intravenous fluid resuscitation, vasopressor infusion, >2 units of packed red blood cell transfusion, endotracheal intubation, advanced cardiac life support, or emergent surgical intervention. Minor clinical events included any diagnosis that required initiation of treatment not included in the definition of acute life threatening clinical events above. False alerts were said to have occurred when no evidence was found that the patient was clinically deteriorating in temporal relationship to lactate DSS activation, or within 72 hours. The positive predictive value of the system was calculated for the real-time detection of acute life-threatening clinical events. Microsoft Research and VassarStats^® on-line statistical software were used for statistical calculations.

Results

8,867 adult patients were admitted during our three-month study period. One hundred and ninety-six of 8867 patients (2.2% 95%CI: 1.9-2.5%) died while in the hospital. Seventy percent (138/196) of these inpatient deaths occurred in the 16% (1400/8867) of patients who triggered a lactate DSS response.

Four hundred seventy-nine of 1400 patients who triggered the lactate DSS already had a clinician-ordered lactate. A DSS-generated order for plasma lactate was entered for the remaining 921 patients, but clinicians cancelled 337 of these. DSS-generated lactate orders were resulted for the remaining 584 patients. These patients were merged with 479 patients who had clinician –ordered lactates for the purposes of further analysis after confirmation that mortality did not depend on how the lactate was ordered (Figure 1).

Figure 1. Stratification of inpatients into five subgroups by the lactate DSS.

Patients who did not trigger the lactate DSS logic (n=7467) had a mortality rate of 0.78% (95%CI: 0.58-0.98). Patients who triggered the lactate DSS and for whom a DSS-generated lactate order was cancelled by the clinician (n=337) had mortality of 2.7% (95%CI: 1.0-4.4%). Patients who triggered the lactate DSS and had a lactate concentration in the normal range (< 2.2 mmol/L; n=721) had mortality of 7.9% (95%CI: 6.0-10.1%), and those with elevated lactates of 2.2-3.9 and >4.0 mmol/L (n=247 and n=95) had mortality rates of 13.0% (95%CI: 9.0-17.8%) and 42.1% (95%CI: 32.0-52.4%) respectively (Figure 2).

Figure 2. Inpatient mortality rates (Y-axis: Percent mortality) with 95% confidence intervals for five subgroups of patients stratified by lactate DSS.

The mortality of patients who triggered a lactate DSS response and for whom a lactate concentration was resulted did not depend on whether the order was DSS-generated or entered by the clinician (13.0% versus 12.1% (P=0.71)). Clinician-entered lactate orders were closely temporally related to the onset of organ dysfunction, preceding lactate DSS triggering by < six hours in 52%, <12 hours in 64%, and <24 hours in 75% of cases. Likelihood ratios for mortality in subgroups of patients with lactates <2.2, 2.2-3.9, and >4.0 mmol/L were 6.1 (95%CI: 5.4-6.9), 11.8 (95%CI: 9.5-14.7), and 32.4 (95%CI: 22.0-47.1) respectively.

Five-strata of mortality risk generated by the lactate DSS yielded an AUROC of 0.80 (95% CI: 0.76-0.84) (Figure 3).

Figure 3. Receiver-operating characteristic curve for mortality risk stratification by the lactate DSS.  

Focused chart review was performed on 61 patients who had elevated lactate (>2.2 mmol/L) detected by a DSS-generated lactate order. Thirty-three (54%) were experiencing acute life-threatening clinical events at the time the lactate DSS was triggered. These included 18 episodes of sepsis. Sepsis was due to pneumonia in nine patients, catheter-associated blood stream infection, bowel perforation, cellulitis, ascending cholangitis, endocarditis, liver abscess, cholecystitis, perianal abscess, or an unidentified source. Other acute life threatening clinical events included five cases of acute gastrointestinal hemorrhage, three of acute respiratory failure, and one each of post-operative bleeding, cardiogenic shock, acute liver failure, retroperitoneal bleeding, acute myocardial infarction, subdural hematoma, and cerebral dural sinus thrombosis. Twenty-one (64%) of these events occurred outside the intensive care unit. The positive predictive value of the detection of SIRS, organ dysfunction and elevated lactate by the lactate DSS for acute life-threatening clinical events was 54% (95%CI: 41.5-66.5%).

Ten minor clinical events included anemia, atrial fibrillation, post-op third spacing, transient mild hypotension associated with end stage liver disease, sedation related to narcotics, and dialysis disequilibrium. There were 18 false alerts among patients with SIRS, organ dysfunction and elevated lactate detected by the system. (18/61=29%).

Discussion

Our lactate DSS effectively segregated a population of adult inpatients into five subgroups with increasing inpatient mortality. Clinician engagement was critically important in achieving this result. About a quarter (337/1400) of patients who triggered the lactate DSS (simultaneously exhibited SIRS and organ dysfunction) were doing well enough in their clinician’s opinion that the DSS-generated lactate order was cancelled. Clinicians exercised good judgment in this regard, identifying a subgroup of patients with inpatient mortality rate not significantly higher than the overall mortality of all patients admitted during the study. This supports our decision to incorporate clinician judgment in our risk stratification method.

Approximately half of patients (721/1400) who triggered the lactate DSS turned out to have a normal lactate concentration, yet suffered inpatient mortality ten-times higher than patients who did not trigger the system. This likely represents the independent association between SIRS and organ dysfunction with the risk for mortality (27, 31,32).

One hundred twenty-nine patients over 3 months (14.5 per 1000 patient admissions) triggered the lactate DSS and were found to have an elevated lactate concentration because of a DSS-generated lactate order. These patients had >50% probability of experiencing an acute life-threatening clinical event at the time the lactate DSS was triggered, and subsequently suffered 50% inpatient mortality.

Our lactate DSS is consistent with the new definition of sepsis because it uses organ dysfunction in addition to SIRS criteria (7). As stated in the new definition of sepsis, “Nonspecific SIRS criteria such as pyrexia or neutrophilia will continue to aid in the general diagnosis of infection” (7). Although these criteria are nonspecific, they appear to be relatively sensitive for sepsis (7,27). Our lactate DSS has excellent discriminant accuracy for predicting inpatient mortality (AUROC=0.80). It is comparable to other criteria such SOFA (AUROC = 0.74) and the Logistic Organ Dysfunction System (AUROC=0.75).The five strata into which it segregates patients could further translate into a decision support-guided treatment protocol, directing appropriate real-time interventions such as those proposed in Table 2.

Table 2. Proposed stratified clinical response to lactate DSS.

* Our data indicate that RRT activation would occur about twice a week at our hospital.

Our lactate DSS is different than EWSs because it specifically prompts assessment of plasma lactate in patients exhibiting SIRS and organ dysfunction, rather than simply generating a warning. But a discussion of the operating characteristics of previously reported EWSs is useful for purposes of comparison. A review of 33 EWSs has reported AUROCs ranging from 0.66-0.78 (19). Several more recent EWSs reported AUROCs of 0.81-0.88 (23,24,26,33), but AUROC comparisons are confounded by lack of consensus regarding which clinical outcome to analyze. Authors have variously chosen 24-hour mortality, ICU transfer, and cardiac arrest, among other outcomes (20,23,24). Many EWSs yield highly stratified results, which may increase the AUROC by adding detail to the shape of the ROC curve, but this will not improve clinical discrimination unless each resulting strata has a distinct clinical response. If a EWS is simply used to activate a rapid response team (RRT), the clinically-achievable discriminant accuracy is best described by a polygonal AUROC derived from a single cutoff with two resulting strata (activate the RRT, or do not activate the RRT). This two-strata AUROC will invariably be lower than the highly stratified AUROC that many authors report (23,24,26,33). Our AUROC analysis is based on 5 strata, each of which could reasonably trigger a distinct clinical response (Table 2).

Our lactate decision support system has a positive predictive value (PPV) for acute life-threatening clinical events that is superior to that of our previous “sepsis alert” (27) and to those reported in several reviews of EWSs. One review of 39 EWSs reported PPVs ranging from 13.5-26.1% (34), and another review of 25 systems reported a median PPV of 36.7% with interquartile range 29.3-43.8% (34). PPV was not reported for several of the most elegant and well-studied EWSs (22,23,25,32). From the perspective of bedside clinicians and rapid response team members, the efficiency of an alert system is strongly influenced by the PPV, because a poor PPV translates to frequent false alerts. The PPV is of particularly concern when the pretest probability of the outcome of interest is low, as in the case of inpatient mortality (2% at our hospital). Bayes theory indicates that a test with relatively good AUROC will have a poor PPV if the pretest probability is low enough.

Our study has several limitations. Our sample size is small compared to many contemporary EWS studies. We did not have the resources to perform focused chart reviews on all study patients and therefore had to limit individual case analysis to a subgroup of study patients. Our simple treatment of vital sign abnormalities as markers of SIRS is not as elaborate as in many EWSs. Our study is only hypothesis-generating, whereas several EWSs are well validated (25,32). We cannot provide data on how our alert might change bedside interventions by clinicians. To our knowledge, no study to date has proven that using a computerized decision support system or EWS to trigger rapid clinical intervention actually improves patient outcomes.

Conclusions

We developed an automated decision-support system that prompts assessment of plasma lactate concentration in patients exhibiting SIRS and organ dysfunction. Our lactate decision support system is different than previously-described EWSs because it engages the clinician in decision-making and incorporates clinical judgment into risk stratification. This system has favorable operating characteristics for the prediction of inpatient mortality and for detecting acute life-threatening events in real time. We have proposed a stratified clinical response based on classification of patients into five subgroups by this system that requires further testing, but our current study was not designed to demonstrate a benefit on clinical outcomes. Our lactate DSS has the potential to improve sepsis bundle compliance by helping clinicians appropriately order lactate concentrations in patients deteriorating due to the onset of sepsis – a hypothesis we are currently investigating. It also has potential for easy generalizability, particularly to other healthcare systems that share the same EMR as ours, but requires further refinement and validation.

Author Contributions

All authors were involved in conceptualization, design and implementation of the decision support system described in this manuscript, and in preparation of the manuscript, and all approve of the content of the manuscript and vouch for the validity of the data. We list below additional contributions from several of the authors:

RAR: data analysis and interpretation, main author of initial draft of the manuscript.

HOW: data analysis and interpretation, contribution to discussion/conclusions

HK: directly in charge of design and pilot implementation team for the decision support system, data interpretation, contribution to discussion, conclusions

RHG: data interpretation, contribution to discussion, conclusions

SCC: data analysis and interpretation, contribution to discussion, conclusions. Manuscript editing.

MM: data collection and analysis

BS: data collection and analysis

References

1. Center for Disease Control. Trends in inpatient hospital deaths: National Hospital Discharge Survey: 2000:2010. NCHS Data Brief 118; 2013. [PubMed]
2. Liu V, Escobar G, Greene JD, Soule J, et al. Hospital deaths in patients with sepsis from two independent cohorts. JAMA. 2014;312:90-92. [CrossRef] [PubMed]
3. Nichols L, Chew B. Causes of sudden unexpected death of adult hospital patients. J Hosp Med. 2012;7:706-8. [CrossRef] [PubMed]
4. McGloin H, Adam SK, Singer M. Unexpected deaths and referrals to intensive care of patients on general wards. Are some cases potentially avoidable? J R Coll Physicians Lond. 1999;33:255-9. [PubMed]
5. Rivers E, Nguyen B, Havstad S, Ressler J, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. NEJM. 2001;345:1368-77. [CrossRef] [PubMed]
6. Dellinger RP, Levy MM, Rhodes A, Annane D, et al. Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41:580. [CrossRef] [PubMed]
7. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016 Feb 23;315(8):801-10. [CrossRef] [PubMed]
8. National Patient Safety Agency. Safer care for the acutely ill patient: learning from serious incidents. 2007;Report # PSO/5. Available online at: http://www.nrls.npsa.nhs.uk/resources/?EntryId45=59828 (accessed 5/9/17).
9. Gultepe E, Green JP, Nguyen H, Adams J, et al. From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system. J Am Med Inform Assoc. 2014;21:315-325. [CrossRef] [PubMed]
10. Jansen TC, van Bommel J, Woodward R, Mulder PG, Bakker J. Association between blood lactate levels, sequential organ failure assessment sub-scores, and 28-day mortality during early and late intensive care unit stay: a retrospective observational study. Crit Care Med. 2009;37:2369-74. [CrossRef] [PubMed]
11. Bakker J, Gris P, Coffernils M, Kahn RJ, Vincent JL. Serial blood lactate levels can predict the development of multiple organ failure following septic shock. Am J Surg. 1996;171:221-6. [CrossRef] [PubMed]
12. Jansen TC, van Bommel J, Bakker J. Blood lactate monitoring in critically ill patients: a systematic health technology assessment. Crit Care Med. 2009;37:2827-39. [CrossRef] [PubMed]
13. Guyette F, Suffoletto B, Castillo JL, Quintero J, Callaway C, Puyana JC. Prehospital serum lactate as a predictor of outcomes in trauma patients: A retrospective observational study. J Trauma. 2011;70:782–6. [CrossRef] [PubMed]
14. Vandromme MJ, Griffin RL, Weinberg JA, Rue LW 3rd, Kerby JD. Lactate is a better predictor than systolic blood pressure for determining blood requirement and mortality: Could prehospital measures improve trauma triage? J Am Coll Surg. 2010;210:861–9. [CrossRef] [PubMed]
15. Jansen TC, van Bommel J, Schoonderbeek FJ, Sleeswijk Visser SL, et al. Early lactate-guided therapy in intensive care unit patients: a multicenter, open-label, randomized controlled trial. Am J Respir Crit Care Med. 2010;182:752-61. [CrossRef] [PubMed]
16. Levy MM, Fink MP, Marshall JC, Abraham E, et al. International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250. [CrossRef] [PubMed]
17. Gao R, Melody T, Daniels DF, Giles S and Fox S. The impact of compliance with 6-hour and 24-hour sepsis bundles on hospital mortality in patients with severe sepsis: a prospective observational study. Crit Care. 2005;9:R764–R770. [CrossRef] [PubMed]
18. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med. 2010; 36: 222–31. [CrossRef] [PubMed]
19. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted "track and trigger" systems. Resuscitation. 2008;77:170-9. [CrossRef] [PubMed]
20. Hodgetts TJ, Kenward G, Vlachonikolis IG, Payne S, Castle N. The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team. Resuscitation. 2002;54:125-31. [CrossRef] [PubMed]
21. Kho A, Rotz D, Alrahi K, Cardenas W, et al. Utility of commonly captured data from an HER to identify hospitalized patients at risk for clinical deterioration. AMIA 2007 symposium proceedings. 404-8.[CrossRef]
22. Howell MD, Donnino M, Clardy P, Talmor D, Shapiro NI. Occult hypoperfusion and mortality in patients with suspected infection. Intensive Care Med. 2007;33:1892-9. [CrossRef] [PubMed]
23. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, et al. Early detection of impending physiological deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7:388-95. [CrossRef] [PubMed]
24. Prytherch DR, Smith GB, Schmidt P, Featherstone PI. ViEWS – towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81:932-7. [CrossRef] [PubMed]
25. Bailey TC, Yixin C, Mao Y, Lu C, et al. A trial of a real-time alert for clinical deterioration in patients hospitalized on general medical wards. J Hosp Med. 2013;8:236-42. [CrossRef] [PubMed]
26. Churpek MM, Yuen TC, Winslow C, Robicsek AA, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Resp Crit Care Med. 2014;190:649-55. [CrossRef] [PubMed]
27. Raschke RA, Owen-Reece H, Khurana H, Groves RH Jr, et al. Clinical performance of an automated systemic inflammatory response syndrome (SIRS)/organ dysfunction alert: a system-based patient safety project. Southwest J Pulm Crit Care. 2014;9:223-9. [CrossRef]
28. Khurana, H, Groves RH, Simons MP, Martin M, Stoffer B, et al. Real-time automated continuous sampling of electronic medical records predicts hospital mortality. Am J Med. 2016 Jul;129(7):688-698.e2. [CrossRef] [PubMed]
29. Jo S, Lee JB, Jin YH, Jeong TO, et al. Modified early warning score with rapid lactate level in critically ill medical patients: the ViEWS-L score. Emerg Med J. 2013;30:123-9. [CrossRef] [PubMed]
30. Rangel-Frausto MS, Pittet, D, Costigan M, et al. The natural history of the Systemic Inflammatory Response Syndrome (SIRS): A prospective study. JAMA. 1995;273:117-123. [CrossRef] [PubMed]
31. Matthew M, Churpek F, Zadravecz J, et al. Incidence and prognostic value of the Systemic Inflammatory Response Syndrome and organ dysfunctions in ward patients. Am J Resp Crit Care Med. 2015;192:958-64. [CrossRef] [PubMed]
32. Pittet D, Range-Frausto S, Tarara LN, Lin N, et al. Systemic inflammatory response syndrome, sepsis, severe sepsis and septic shock: incidence, morbidities and outcomes in surgical ICU patients. Intensive Care Med. 1995;21:302-9. [CrossRef] [PubMed]
33. Kellett J, Kim A. Validation of an abbreviated Vitalpac early warning score (ViEWS) in 75,419 consecutive admission to a Canadian regional hospital. Resuscitation. 2012;83:297-302. [CrossRef] [PubMed]
34. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, et al. A review and performance evaluation of single-parameter "track and trigger" systems. Resuscitation. 2008;79:11-21. [CrossRef] [PubMed]
35. Gao H, McDonnell A, Harrison DA, Moore T, et al. Systemic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007;33:667-79. [CrossRef] [PubMed]

Cite as: Raschke RA, Khurana H, Owen-Reece H, Groves RH Jr, Curry SC, Martin M, Stoffer B. Clinical performance of an interactive clinical decision support system for assessment of plasma lactate in hospitalized patients with organ dysfunction. Southwest J Pulm Crit Care. 2017;14:241-52. doi: https://doi.org/10.13175/swjpcc058-17 PDF