Rapidly Fatal COVID-19-associated Acute Necrotizing Encephalopathy in a Previously Healthy 26-year-old Man

Robert A. Raschke MD and Cristian Jivcu MD

HonorHealth Scottsdale Osborn Medical Center

Scottsdale, AZ USA

Case Presentation

A 26-year-old man presented to our Emergency Department at 0200 on the day of admission with chief complaints of subjective fever, leg myalgias, and progressive dyspnea of one week duration. An oropharyngeal swab PCR had revealed SARS-CoV-2 RNA three days previously. He had not received a SARS CoV-2 vaccination, but had made an appointment to receive it just a few days prior to the onset of his symptoms.

The patient had no significant past medical history, was taking no medications except for ibuprofen and acetaminophen over the past week, and did not take recreational drugs. He specifically denied headache and had no prior history of seizure.

On admission, his HR was 150 bpm (sinus), RR 22, BP 105/46 mmHg, temp 40.2° C. and SpO₂ 92% on room air. He was ill-appearing, but alert and oriented, his neck was supple and lung auscultation revealed bilateral rhonchi, but physical examination was otherwise unremarkable.

A CBC showed WBC 17.3 10³/uL, hemoglobin 13.9 g/dl, and platelet count 168 K/uL. A complete metabolic profile was normal except for the following: Na 135 mmol/L, creatinine 1.7 mg/dL, AST 95 and ALT 134 IU/L. D-dimer was 1.08 ug/ml (normal range 0.00-0.50 ug/ml), and ferritin 783 ng/ml. A urine drug screen was negative. Chest radiography showed subtle bilateral pulmonary infiltrates. CT angiography of the chest was negative for pulmonary embolism but showed bilateral patchy infiltrates consistent with COVID19 pneumonia. One liter NS bolus and dexamethasone 10mg were given intravenously, acetaminophen administered orally, and the patient was admitted to telemetry.

Shortly thereafter, the patient experienced a brief generalized seizure associated with urinary incontinence. He was stuporous post-ictally, exhibiting only arm flexion to painful stimuli. A stroke alert was called and radiographic studies emergently obtained. CT of the brain was normal and CT angiography of the head and neck showed no large vessel occlusion or flow-limiting stenosis, and a CT perfusion study (Figure 1) showed patchy Tmax prolongation in the right cerebellum and bilateral parietal occipital lobes “which may reflect artifact or relative ischemia” with no matching core infarct.

Figure 1. CT perfusion study showing mild bilateral posterior distribution ischemia (Tmax > 6 secs) without matching core infarct (CBF<30%), interpreted by a neuroradiologist as possible artifact.

The patient was transferred to the ICU at 10:00, and experienced a 40-second generalized tonic-clonic seizure shortly thereafter. Lorazepam 2mg was administered intravenously. The HR was 104, RR 21, BP 105/61, temp 36.5 C. and SpO₂ 96% on 2L /min nasal canula oxygen. On neurological examination, the Glasgow Coma Scale was 3, right pupil was 3mm, left pupil 2mm - both reactive, the gaze was disconjugate and directed downward, there was no blink to visual threat, and glabellar ridge pressure did not elicit grimace, but minimal arm flexion. The gag reflex was positive. Peripheral reflexes were 2+ with down-going toes bilaterally. Levetiracetam 1000mg bolus was administered intravenously. Glucose was 147 mg/dL. An EEG obtained at 12:00 showed diffuse bilateral slowing without seizure activity. A presumptive diagnosis of post-ictal encephalopathy was made. The patient seemed to be protecting his airway and nasal canula oxygen was continued.

The patient’s condition was not noted to significantly change over the next 12 hours. There were no episodes of hypoxia, hypotension or hypoglycemia. Around 0100 on the second day of hospitalization, the patient exhibited extensor-posturing and appeared to be choking on his oral secretions. HR rose to 135, BP 155/99, RR 12 and temp 37.8 C. His SpO₂ fell into the mid 80% range. He no longer had a gag or cough reflex and he was emergently intubated without complication. MRI (Figure 2) and MRV of the brain were emergently obtained. 

Figure 2. A: T2-weighted image demonstrating bilateral thalamic and L occipital white matter hypoattenuation. B: DWI and GRE images showing bilateral thalamic infarctions with hemorrhage. C: Representative DWI images of cerebrum and cerebellum and pons showing widespread diffusion restriction.

The MRI showed extensive diffusion restriction involving bilateral thalami, cerebellar hemispheres, pons, and cerebral hemispheres with scattered hemorrhage most obvious/confluent in the bilateral thalami.

Normal flow voids were present in intracranial arteries and venous structures. Partial effacement of the lateral and third ventricles was noted, with early uncal herniation. The MRV showed no evidence of dural venous sinus thrombosis.

At 05:00 of the second hospital day, it was noted that the patient’s pupils were dilated and unreactive and his respiratory rate was 16 – equal to the respiratory rate set on the ventilator. BP fell to 85/45 and norepinephrine infusion was started to maintain MAP >65 mmHg. STAT CT brain (Figure 3) showed hemorrhagic infarcts of the bilateral thalami with surrounding edema, interval development of low attenuation of the bilateral cerebrum and cerebellum, and mass effect with total effacement of fourth ventricle, basal cisterns and cerebral sulci consistent with severe cerebral edema.

Figure 3. STAT CT brain from 05:30 on the second hospital day showing bilateral thalamic infarctions and diffuse cerebral edema with effacement of the sulci and loss of grey/white differentiation.

Two neurologists confirmed the clinical diagnosis of brain death, including an apnea test. A venous ammonia level ordered that morning was not drawn. An autopsy was requested by the physicians, but not able to be obtained.

Discussion

Acute necrotizing encephalopathy (ANE) is a rarely-reported clinical-radiographic syndrome lacking pathopneumonic laboratory test or histological findings (1-3). It is characterized by an acute febrile viral prodrome, most commonly due to influenza or HHV-6, followed by rapidly progressive altered mental status and seizures. The most specific finding of ANE is necrosis of the bilateral thalami, appearing on MRI as hypoattenuated lesions on T2 and FLAIR images with diffusion restriction on DWI, and often with hemorrhage demonstrated on GRE images (as shown in figure 2 above). Symmetric multifocal lesions are typically seen throughout various other locations in the brain including the cerebral periventricular white matter, cerebellum, brainstem and spinal cord. Mizuguchi (who first described ANE in 1995) proposed elevation of serum aminotransferase without hyperammonemia, and cerebrospinal albuminocytologic dissociation (elevated CSF protein without leukocytosis) as laboratory criteria supporting the diagnosis of ANE (1,2). These were only partially evaluated in our patient. The mortality of ANE is 30% and significant neurological sequelae are common in survivors (2).

The clinical, radiographic and laboratory findings in our case are all characteristic of ANE, but our work-up was abbreviated by the patient’s fulminant presentation. The differential diagnosis includes hyper-acute forms of acute disseminated encephalomyelitis (ADEM) or acute hemorrhagic leukoencephalitis that may also occur after a viral prodrome and may be associated with diffuse white matter lesions (4,5), although bilateral thalamic necrosis is not characteristic of either of these entities. Examination of cerebral spinal fluid (CSF) for pleocytosis, oligoclonal bands, and testing for the myelin oligodendrocyte glycoprotein IgG autoantibody and the aquaporin-4 IgG serum autoantibody would have been indicated to further evaluate for the initial presentation of a relapsing CNS demyelinating disease (5,6). CSF examination would also have been helpful in ruling out viral encephalitis affecting the thalami, such as that caused by West Nile Virus (WNV) (7). An acute metabolic encephalopathy with diffuse brain edema, such as that caused by severe hyperammonemia associated with late-onset ornithine transcarbamylase deficiency (8) was not ruled out. Arterial or venous thromboembolism associated with COVID-19 were effectively ruled out by CT angiogram, CT perfusion and MRI and MRV findings.    

We found five previous case reports of ANE as a complication of COVID-19, ranging 33-59 years of age (9-13). The onset of altered mental status occurred 3, 4, 7,10 and 21 days after onset of COVID-19 symptoms and rapidly progressed to coma. Two had generalized seizures, one myoclonus and another “rhythmic movements” of an upper extremity. All had bilateral hypoattenuation of the thalami on CT and MRI with variable involvement of temporal lobes, subinsular regions, cerebellum, brainstem and supratentorial grey and white matter. Two patients had EEGs that showed generalized slow waves. All underwent examination of CSF with negative PCR tests for various common encephalopathy viruses including herpes simplex virus 1&2 and WNV - four reported CSF protein and cell counts, three of which demonstrated albuminocytologic dissociation. Three patients received IVIG. Two patients died on days 5 and 8 after onset of neurological symptoms. Two recovered after prolonged ICU care and the outcome of the third patient was not reported. ANE may be less rare than these few case reports suggest. A retrospective study carried out at 11 hospitals in Europe describes radiographic findings of 64 COVID-19 patients with neurological symptoms (14). The most common finding was ischemic stroke, but 8 patients had MRI findings consistent with encephalitis and two had findings characteristic of ANE.

The pathogenesis of ANE is unknown. Ten cases of fatal ANE with brain biopsy are reported (1,15-19). These showed diffuse cerebral edema, and hemorrhagic necrosis invariably involving the thalami. An exudative small vessel vasculopathy with endothelial necrosis was found in 7/10 patients (This could perhaps explain the early CT perfusion findings interpreted as artifactual in our patient). Demyelination or inflammatory infiltration of the brain or leptomeninges was absent. There has been conjecture that these pathological findings might be due to disruption of the blood brain barrier caused by hypercytokinemia but there is scant supportive evidence (20). 

There is no proven treatment for ANE. Corticosteroids, IVIg and plasma exchange have been previously used (3,9-11,21). Clinical trials are unlikely given the rarity of the disorder.

It was unfortunate that this young man had not availed himself of SARS CoV-2 vaccination. We did not make a pre-mortem diagnosis of ANE between his first abnormal CT brain at 0100 and his death at 06:00. We would have performed an LP, measured serum ammonia and given a trial of corticosteroids and IVIg if we had had more time.

References

1. Mizuguchi M, Abe J, Mikkaichi K, Noma S, Yoshida K, Yamanaka T, Kamoshita S. Acute necrotising encephalopathy of childhood: a new syndrome presenting with multifocal, symmetric brain lesions. J Neurol Neurosurg Psychiatry. 1995 May;58(5):555-61. [CrossRef] [PubMed]
2. Mizuguchi M. Acute necrotizing encephalopathy of childhood: a novel form of acute encephalopathy prevalent in Japan and Taiwan. Brain Dev. 1997 Mar;19(2):81-92. [CrossRef] [PubMed]
3. Wu X, Wu W, Pan W, Wu L, Liu K, Zhang HL. Acute necrotizing encephalopathy: an underrecognized clinicoradiologic disorder. Mediators Inflamm. 2015;2015:792578. [CrossRef] [PubMed]
4. Marchioni E, Ravaglia S, Montomoli C, et al. Postinfectious neurologic syndromes: a prospective cohort study. Neurology. 2013 Mar 5;80(10):882-9. [CrossRef] [PubMed]
5. Manzano GS, McEntire CRS, Martinez-Lage M, Mateen FJ, Hutto SK. Acute Disseminated Encephalomyelitis and Acute Hemorrhagic Leukoencephalitis Following COVID-19: Systematic Review and Meta-synthesis. Neurol Neuroimmunol Neuroinflamm. 2021 Aug 27;8(6):e1080. [CrossRef] [PubMed]
6. López-Chiriboga AS, Majed M, et al. Association of MOG-IgG Serostatus With Relapse After Acute Disseminated Encephalomyelitis and Proposed Diagnostic Criteria for MOG-IgG-Associated Disorders. JAMA Neurol. 2018 Nov 1;75(11):1355-1363. [CrossRef] [PubMed]
7. Guth JC, Futterer SA, Hijaz TA, Liotta EM, Rosenberg NF, Naidech AM, Maas MB. Pearls & oy-sters: bilateral thalamic involvement in West Nile virus encephalitis. Neurology. 2014 Jul 8;83(2):e16-7. [CrossRef] [PubMed]
8. Cavicchi C, Donati M, Parini R, et al. Sudden unexpected fatal encephalopathy in adults with OTC gene mutations-Clues for early diagnosis and timely treatment. Orphanet J Rare Dis. 2014 Jul 16;9:105. [CrossRef] [PubMed]
9. Poyiadji N, Shahin G, Noujaim D, Stone M, et al.  COVID19-associated acute necrotizing encephalopathy: CT and MRI features.  Radiology. 2020;296:E119-E120. [CrossRef]
10. Virhammar J, Kumlien E, Fällmar D,et al. Acute necrotizing encephalopathy with SARS-CoV-2 RNA confirmed in cerebrospinal fluid. Neurology. 2020 Sep 8;95(10):445-449. [CrossRef] [PubMed]
11. Delamarre L, Galion C, Goudeau G, et al. COVID-19-associated acute necrotising encephalopathy successfully treated with steroids and polyvalent immunoglobulin with unusual IgG targeting the cerebral fibre network. J Neurol Neurosurg Psychiatry. 2020 Sep;91(9):1004-1006. [CrossRef] [PubMed]
12. Dixon L, Varley J, Gontsarova A, Mallon D, Tona F, Muir D, Luqmani A, Jenkins IH, Nicholas R, Jones B, Everitt A. COVID-19-related acute necrotizing encephalopathy with brain stem involvement in a patient with aplastic anemia. Neurol Neuroimmunol Neuroinflamm. 2020 May 26;7(5):e789. [CrossRef] [PubMed]
13. Elkady A, Rabinstein AA. Acute necrotizing encephalopathy and myocarditis in a young patient with COVID-19. Neurol Neuroimmunol Neuroinflamm Sep 2020, 7 (5) e801. [CrossRef]
14. Kremer S, Lersy F, Anheim M, et al. Neurologic and neuroimaging findings in patients with COVID-19: A retrospective multicenter study. Neurology. 2020 Sep 29;95(13):e1868-e1882. [CrossRef] [PubMed]
15. Kirton A, Busche K, Ross C, Wirrell E. Acute necrotizing encephalopathy in caucasian children: two cases and review of the literature. J Child Neurol. 2005 Jun;20(6):527-32. [CrossRef] [PubMed]
16. Mastroyianni SD, Gionnis D, Voudris K, Skardoutsou A, Mizuguchi M. Acute necrotizing encephalopathy of childhood in non-Asian patients: report of three cases and literature review. J Child Neurol. 2006 Oct;21(10):872-9. [CrossRef] [PubMed]
17. Nakano I, Otsuki N, Hasegawa A. Acute Stage Neuropathology of a Case of Infantile Acute Encephalopathy with Thalamic Involvement: Widespread Symmetrical Fresh Necrosis of the Brain. Neuropathology 1993;13: 315-25. [CrossRef]
18. Yagishita A, Nakano I, Ushioda T, Otsuki N, Hasegawa A. Acute encephalopathy with bilateral thalamotegmental involvement in infants and children: imaging and pathology findings. AJNR Am J Neuroradiol. 1995 Mar;16(3):439-47. [PubMed]
19. San Millan B, Teijeira S, Penin C, Garcia JL, Navarro C. Acute necrotizing encephalopathy of childhood: report of a Spanish case. Pediatr Neurol. 2007 Dec;37(6):438-41. [CrossRef] [PubMed]
20. Wang GF, Li W, Li K. Acute encephalopathy and encephalitis caused by influenza virus infection. Curr Opin Neurol. 2010 Jun;23(3):305-11. [CrossRef] [PubMed]
21. Okumura A, Mizuguchi M, Kidokoro H, et al. Outcome of acute necrotizing encephalopathy in relation to treatment with corticosteroids and gammaglobulin. Brain Dev. 2009 Mar;31(3):221-7. [CrossRef] [PubMed]

Cite as: Raschke RA, Jivcu C. Rapidly Fatal COVID-19-associated Acute Necrotizing Encephalopathy in a Previously Healthy 26-year-old Man. Southwest J Pulm Crit Care. 2021;23(5):138-43. doi: https://doi.org/10.13175/swjpcc039-21 PDF

July 2020 Critical Care Case of the Month: Not the Pearl You Were Looking For...

Yuet-Ming Chan MD¹

David C. Miller MD²

Farshad Shirazi MD³

Janet Campion MD²

¹Department of Medicine, ²Pulmonary, Allergy, Critical Care and Sleep Medicine and ³Arizona Poison and Drug Information Center

University of Arizona School of Medicine

Tucson, AZ USA

History of Present Illness

A 75-year-old man presented with unsteady gait, difficulty concentrating and abdominal pain with loose stools. One day prior to admission, he experienced waxing and waning nausea, cramping abdominal pain, one episode of emesis and loose stools. He described acute gait disorder related to difficulty with balance. Due to concern for dehydration, he drank 10-12 cans of carbonated water without further emesis. He also experienced vague and alternating sensations of feeling “hot” in half of his body and “cold” in the other half of his body. Forty-eight hours prior to presentation, he had just returned from a five-day trip to New Orleans.

PMH, SH, and FH

The patient has hypertension and hyperlipidemia that is well-controlled. Regular medicines include losartan, diltiazem, HCTZ and simvastatin. He is a professor of medicine. He had distant tobacco use with a 10 pk-yr history. He denies recreational drug use. He endorsed drinking one glass of wine per day during his recent trip. He had eaten oysters and redfin fish during his trip.

Physical Examination

• Afebrile, HR=38, RR=12, BP=134/72, O2 sat=95% on RA
• In general, patient was slightly argumentative and in obvious distress due to abdominal pain. HEENT - nonicteric, pupils reactive, moist oral mucosa
• Neck - No elevated JVP, LAD or thyromegaly
• CV - Bradycardic, regular, no murmur
• Pulmonary - Clear to auscultation all lung fields
• Abdomen - Soft with diffuse tenderness to palpation, bowel sounds present, no HSM or mass
• Lower extremities - Cool to the touch without cyanosis, intact and symmetric distal pulses
• Neuro – Cranial nerves intact, no focal motor or sensory deficits, oriented but with difficulty concentrating on thoughts, poor short-term recall, no obvious visual or auditory hallucinations.

Laboratory

Initial laboratory testing was notable for hyponatremia of 126, otherwise a metabolic panel, complete blood count, troponin, urinalysis, urine drug screen and thyroid stimulating hormone were unremarkable. EKG showed sinus bradycardia without ischemic changes. An abdominal flat plate (KUB) showed a nonspecific bowel gas pattern without evidence of obstruction. Chest x-ray was negative for acute cardiopulmonary abnormality.

He was given 1 liter of normal saline with improvement of sodium to 131, but his pulse remained low at 36. He also developed worsening nausea and mentation, was incoherent at times, and began telling staff that “I’m going to die.”

For the initial presentation of nausea, vomiting, bradycardia, hyponatremia, mental status changes, what is your leading diagnosis?

1. Acute porphyria
2. Excessive water intake
3. Neurotoxic shellfish poisoning
4. Recreational drug use
5. Small cell lung cancer

Cite as: Chan Y-M, Miller DC, Shirazi F, Campion J. July 2020 critical care case of the month: not the pearl you were looking for. Southwest J Pulm Crit Care. 2020;21(1):1-8. doi: https://doi.org/10.13175/swjpcc002-20 PDF

The Explained Variance and Discriminant Accuracy of APACHE IVa Severity Scoring in Specific Subgroups of ICU Patients

Robert A Raschke MD^1,2

Richard D Gerkin MD¹

Kenneth S Ramos MD^1,2

Michael Fallon MD²

Steven C Curry MD^1,2

Division of Clinical Data Analytics and Decision Support and the Department of Medicine

University of Arizona College of Medicine-Phoenix.

Phoenix, AZ USA

(Click here for accompanying editorial)

Abstract

Objective: The Acute Physiology and Chronic Health Evaluation (APACHE) is a severity scoring system used to predict healthcare outcomes and make inferences regarding quality of care. APACHE was designed and validated for use in general ICU populations, but its performance in specific subgroups of ICU patients is unproven. Quantitative performance referents for severity scoring systems like APACHE have not been established. This study compares the performance of APACHE IVa in several common subgroups of ICU patients to the performance of APACHE IVa and a referent scoring system applied in a general ICU population.

Design: Observational cohort.

Setting: Seventeen ICUs.

Patients: Adult patients meeting criteria for APACHE IVa scoring.

Intervention: We designed a “two-variable severity score” (2VSS) to provide “weak” reference values for explained variance (R²) and discriminant accuracy to use in our comparisons. R² and AUROC were calculated for 2VSS and APACHE IVa outcome predictions in the overall cohort, and for APACHE IVa in subgroups with sepsis, acute myocardial infarction, coronary artery bypass grafting, stroke, gastrointestinal bleeding, trauma, or requiring mechanical ventilation. APACHE IVa subgroup performance was compared to APACHE VIa and 2VSS performance in the overall cohort.  

Measurements and Main Results: APACHE IVa out-performed 2VSS in our cohort of 66,821 ICU patients (R²: 0.16 vs 0.09; AUROC: 0.89 vs 0.77). However, APACHE IVa performance was significantly diminished in subgroups with sepsis, coronary artery bypass grafting, gastrointestinal bleeding or requiring mechanical ventilation compared to its performance in the overall cohort analysis. APACHE IVa performance in patients undergoing CABG (R²: 0.03, AUROC: 0.74) failed to surpass 2VSS performance referents.

Conclusions:  The performance of severity scoring systems like APACHE might be insufficient to provide valid inferences regarding quality of care in select patient subgroups. Our analysis of 2VSS provides quantitative referents that could be useful in defining acceptable performance.

Introduction

The Acute Physiology and Chronic Health Evaluation (APACHE) has undergone iterative refinement over the past 40 years and is currently the most widely used severity scoring system in the United States (1-3). APACHE provides a score based on the patient’s age, vital signs and laboratory values on the first ICU day and chronic health conditions. This score is used in combination with the patient’s admission diagnosis and other information to calculate predicted hospital and ICU mortality and length-of-stay (LOS), and days of mechanical ventilation. Ratios derived from these calculations, such as the standardized mortality ratio (observed/predicted mortality) and observed/predicted LOS are used by the Centers for Medicare and Medicaid Services, managed care plans, health insurance plans and consumers to benchmark and compare the quality of care provided by physicians, hospitals and healthcare systems. APACHE was updated and revalidated using large clinical databases in 2001-2003, yielding APACHE version IV (1,2) and in 2006-2008, yielding APACHE version IVa (4). 

The use of severity scoring systems such as APACHE to make inferences regarding quality of care is susceptible to bias if the regression models employed do not adequately characterize severity of illness. This is a particular liability when applied to a different population of patients than those for whom the system was originally developed and validated (3,5). This is likely because the optimal set of predictor variables in a severity scoring system is specific to the patient population of interest. The optimal predictor variables for patients with pneumococcal pneumonia might include factors such as prior pneumococcal exposure history, the specific competency of the patient’s immune response against pneumococcus, ciliary function of the lower respiratory tract, current cardiopulmonary capacity, and bacterial virulence factors.  The optimal set of specific predictor variables in patients with stroke or trauma are likely quite different. APACHE uses a set of predictor variables empirically found to be predictive in heterogeneous populations of general ICU patients, but these may not necessarily provide acceptable severity-adjustment for specific subpopulations of ICU patients.

The performance of severity scoring systems is typically assessed using statistical tests that include Pearson’s R-squared (R²) - which describes the “explained variance” of the system for prediction of continuous outcomes like LOS, and the area under the receiver operating curve (AUROC) - which describes the “discriminant accuracy” of the system for prediction of discrete outcomes such as mortality. APACHE IV has yielded an R² of 0.21 for LOS prediction, and AUROC of 0.88 for mortality prediction in a cohort of 131,000 general ICU patients (1,2). However, R² as low as 0.03 and AUROC as low as 0.67 have been reported for APACHE IV outcome predictions in different reference populations, such as those with surgical sepsis (6,7). The performance of the current version, APACHE IVa, is unpublished for many important subgroups of ICU patients.

It has been proposed that AUROC results in the range of 0.70-0.80 indicate “good” discriminant accuracy, and values in the range of 0.80-0.90 are taken to be “very good” or “excellent” (3,8,9), but these subjective ratings have no clear mathematical justification. AUROCs as high as 0.80 have been achieved by scoring systems that utilized only 1-3 predictor variables (10-14). It does not seem plausible that so few variables could acceptably characterize the complex nature of severity-of-illness. R² and AUROC do not have established and well-justified performance thresholds and are therefore of limited value in determining whether a severity scoring system provides valid inferences regarding quality of care.  

Therefore, we first set out to quantify performance thresholds for R² and AUROC by designing a severity score which only incorporated two predictor variables, to intentionally limit the explained variance and discriminant accuracy of the system. This method was previously recommended by the RAND Corporation for assessing severity scoring systems like APACHE because it provides a population-specific referent of unacceptable performance to which the system of interest can be compared (10). We subsequently compared the statistical performance of our two-variable severity score (2VSS) to that of APACHE IVa (which incorporates 142 variables) in a large cohort of ICU patients, and in several common subgroups. Our hypothesis was that APACHE IVa would have diminished and possibly unacceptable explained variance and discriminant accuracy in certain specific subgroups.

Methods

Our Institutional Review Board provided exemption from human research requiring informed consent. Consecutive patients >16 years of age admitted to any ICU in 17 Banner Health acute care hospitals between January 1, 2015 and September 31, 2017 were eligible for inclusion in our cohort of ICU patients. The hospitals ranged from a 44-bed critical access facility to a 708-bed urban teaching hospital in the southwestern United States. The ICUs included general medical-surgical units, as well as specialty-specific cardiovascular, coronary, neurological, transplant and surgical-trauma ICUs. Only the first admission for each patient was included. Patients were excluded if they were admitted as a transfer from another hospital ICU, their ICU LOS was < four hours, or records were missing data required to calculate predicted outcomes using APACHE IVa methodology.

Data used to calculate the acute physiology score (APS) were collected by direct electronic interface between the Cerner Millennium^® electronic medical record and Philips Healthcare Analytics. The worst physiological values occurring during the first ICU day were extracted electronically for Acute Physiology Score (APS) calculation using commercial software provided by the Phillips eICU^® program. Chronic health conditions required for APACHE score calculations and admission information needed for calculation of expected mortality (including admission diagnosis) were entered by nurses who staff our critical care telemedicine service. Observed and predicted ICU and hospital LOS, ventilator days, and ICU and hospital mortality were provided by Philips Healthcare using proprietary APACHE IVa methodology (Cerner Corp. Kansas City, MO).

The 2VSS incorporated only the patient’s age and requirement for mechanical ventilation (yes/no) and used multiple linear regression for prediction of LOS and ventilator days, and multiple logistic regression for prediction of mortality. In contrast, APACHE IVa incorporates 142 variables (27 in the APACHE score, plus 115 admission diagnostic categories) and uses disease-specific regression models serially revised and revalidated in large patient populations (1-3). The two variables incorporated in our 2VSS have been shown to contribute only 10% of the discriminant accuracy of APACHE IV for predicting ICU mortality (1). Therefore, we posited that the best observed AUROC and R² achieved by 2VSS in our cohort analysis could reasonably determine referents of unacceptable performance for comparison with APACHE IVa performance in the analysis of our cohort and in specific subgroups.

Cohort analysis: We used APACHE IVa and the 2VSS to predict five outcomes in our cohort of ICU patients: ICU and hospital LOS, ventilator days, and ICU and hospital mortality. R² was calculated for LOS and ventilator days, and AUROC for mortality outcomes. APACHE IVa results were compared to those of 2VSS. Differences between AUROC results were determined to be statistically significant by comparison of 95% confidence intervals calculated using a nonparametric method based on the Mann-Whitney U-statistic. The highest R² and AUROC achieved by 2VSS in the ICU cohort were used to establish referents of unacceptable performance in all subsequent comparisons.

Subgroup analyses: R² and AUROC were then calculated for APACHE IVa outcome prediction in seven subgroups of ICU patients, including those with admission diagnoses of sepsis, acute myocardial infarction, coronary artery bypass grafting (CABG - with or without other associated cardiac procedures such as valve replacement), stroke, gastrointestinal bleeding, trauma, or requirement of mechanical ventilation. The performance of APACHE IVa in each subgroup was compared to the performance of APACHE IVa and 2VSS in the cohort analysis.

Results

71,094 patients were admitted to study ICUs during the study period. Of these, 2,545 were excluded due to ICU LOS < four hours, 1,379 due to missing data required to calculate APACHE IVa predicted outcomes, and 349 due to transfer from another ICU. The remaining 66,821 patients were included in the analysis. The mean age was 65.7 years (SD 16.3). The most common ICU admission diagnoses were: infections 21.0% (16.8 % due to sepsis); cardiac 14.8% (4.6% due to acute myocardial infarction); cardiothoracic surgery 8.8% (3.8% due to CABG); neurological 8.7% (4.1% due to stroke); pulmonary 7.3%; vascular 5.8%; trauma 5.7%; and gastrointestinal 4.8% (4.0% due to GI bleeds), metabolic/endocrine 4.6%; toxicological 4.5%; cancer 3.8%; and general surgery 3.2%.

Table 1 compares the explained variance (R²) and discriminant accuracy (AUROC) of APACHE IVa and 2VSS outcome predictions in the ICU cohort.

Table 1. Comparison of APACHE IVa to a 2-variable severity score (2VSS) for outcome prediction in a cohort of 66,821 ICU patients. 

Bold font represents the best performance achieved by the 2VSS by R2 and AUROC.

The highest R² achieved by 2VSS was for ICU LOS (R² = 0.09) and the highest AUROC for ICU mortality (AUROC = 0.77).

Subgroup results for APACHE IVa are shown in Table 2.

Table 2. Performance of APACHE IVa outcome prediction in selected subgroups in descending order of discriminant accuracy for ICU mortality. (Click here for enlarged Table 2)

Bold font indicates performance statistically no better than the best performance of 2VSS in the ICU cohort.

*Indicates statistically significantly-reduced performance compared to APACHE IVa in the inclusive ICU cohort (non-overlapping 95% confidence intervals).

Abbreviations: Vent = patients requiring mechanical ventilation; AMI = acute myocardial infarction; GI = gastrointestinal, CABG = coronary artery bypass grafting.

AUROC for APACHE IVa mortality predictions (hospital and ICU mortality) ranged from 0.74-0.90 and were statistically-significantly diminished in subgroups of patients with sepsis, GI bleeds, CABG or mechanical ventilation compared to APACHE IVa performance in the cohort analysis. R² for APACHE IVa prediction of ventilator days was less than 0.09 (the performance referent established by 2VSS) in subgroups of patients with trauma, stroke, acute myocardial infarction, sepsis, GI bleeds and CABG. APACHE IVa predictions of ICU LOS, ventilator days, ICU mortality and hospital mortality for patients who underwent CABG yielded: R² 0.03, R² 0.02, AUROC 0.74 and AUROC 0.75, respectively – all failing to exceed the performance referents established by our cohort analysis by 2VSS.

Discussion

Our study employed empirically-derived, quantitative referents of unacceptable severity-adjustment performance: R² < 0.09 and AUROC < 0.77. APACHE IVa significantly surpassed these referents in all comparisons made in the analysis of our inclusive cohort of ICU patients. R² values for APACHE IVa indicate that it explains about 15- 25% of the variance in hospital and ICU LOS and about 10% of the variance in ventilator days and that it provides discriminant accuracy >0.85 for mortality prediction in this general ICU population. These findings are consistent with previous reports of APACHE IV performance in other large cohorts of ICU patients (1,2,4,15).

However, APACHE IVa performance was significantly diminished in specific subgroups of ICU patients – notably those with sepsis, GI bleeding, requiring mechanical ventilation and undergoing CABG. Values for R² for the prediction of ventilator days in several subgroups were as low as 0.02 – explaining only 2% of the observed variance in ventilator days. Hospital mortality prediction for patients with sepsis yielded an AUROC 0.79 – barely superior to the referent AUROC of 0.77 achieved by 2VSS, and arguably only because of our large sample size. APACHE IVa prediction of ICU LOS, vent days, ICU mortality and hospital mortality in patients undergoing CABG all failed to exceed the performance referents set by 2VSS. 

Few published studies are available to provide meaningful comparisons with the subgroup results from our study. Most describe smaller patient populations outside the U.S. (6,16,17,18). Previous use of APACHE IV to predict outcomes in patients with sepsis reported AUROCs ranging from 0.67 to 0.94 (6,16,19). APACHE IV uses a specific logistic modeling technique and has been specifically validated for CABG patients, but CABG-specific R² and AUROC were not reported (20). No previous study compared APACHE IVa performance in subgroups with that in a general population of ICU patients using quantitative performance referents.

Our findings are important because although general severity scoring systems like APACHE IVa are not optimized for use in specific ICU patient subgroups, they are often used in this manner to make implications regarding quality of care (6,16-19,21-26). In addition to the subgroups discussed above, previous studies have employed general severity scoring system to predict outcomes in subgroups of patients with acute coronary syndrome (17), acute kidney failure (21), malignancy (22), organ transplantation (23), ECMO (24), cardiac surgery (25) and survivors of cardiac arrest (4,26). Many of these studies report AUROCs inferior to our 2VSS referent (6,19,20,23-26). Diagnosis-specific scoring systems, such as the Cardiac Surgery Score (CASUS), generally have provided superior discriminant accuracy in the specific subsets of patients they were designed to serve (27-29).

We believe that general severity scoring systems like APACHE IVa are at an inherent disadvantage in the prediction of outcomes in specific subgroups of ICU patients, because they employ general predictor variables empirically-chosen to work best in heterogeneous patient populations. The APACHE score for example comprises 27 parameters, including vital signs, laboratory values, and specific chronic health items, with a few additional clinical variables added for patients undergoing CABG. As the field of precision medicine has emerged, a rapidly-growing literature describes the use of highly-specific biomarkers, proteomic assays, genomic microarrays and whole-genome sequencing in disease-specific outcome predictions (30-38).  As the science of precision medicine advances, it’s likely that we will develop more precise methods of outcome prediction for specific subgroups of patients that are likely to surpass the performance of general severity scoring systems based only on clinical variables and routine laboratory tests. 

Our study illustrates some features of the explained variance and discriminant accuracy of current severity scoring systems. Our finding that R² does not generally exceed 0.25 is consistent with the findings of other investigators in regards to other well-validated severity scoring systems (2,11,39). This indicates that less than 25% of the between-patient variability in ICU or hospital LOS is explained by current scoring systems. There are two possible explanations for this finding. Either current severity scores are not well-designed to predict LOS, or LOS is inherently not very dependent on severity-of-illness. Our findings imply that ratios of observed/predicted LOS, or observed/predicted ventilator days calculated using current severity scoring systems, may be vulnerable to significant residual bias.  

The differences in the discriminant accuracy achieved by 2VSS and APACHE IVa were surprisingly narrow (e.g., AUROC 0.77 vs. 0.89 for ICU mortality), suggesting that the relationship between AUROC and system complexity is non-linear. We recently performed a Monte Carlo simulation that showed that AUROC increases quadratically in diminishing increments as explanatory power is added to a mortality prediction model, and that the model can achieve an AUROC of 0.85 when only half of important predictor variables have been incorporated (40). This suggests that even the best current severity scoring systems, achieving AUROCs near 0.85, may leave many important aspects of severity-of-illness unaccounted for.

Based on our study results and review of the literature, we suggest that an AUROC ≤ 0.80 represents unacceptable discriminant accuracy in relation to severity scoring systems. This proposition is more conservative than previously-described subjective rating scales (3,8,9), but consistent with published examples of severity scoring systems that are inherently unlikely to yield acceptable discriminant accuracy. Systems incorporating only 1-3 variables have achieved AUROCs of 0.70-0.80, including one intentionally-designed to perform poorly (AUROC 0.70) (10), and others based only on: categorical self-assessment of health (i.e. as poor, good, excellent) (AUROC 0.74) (12), age (AUROC 0.76) (13) or hypotension, tachypnea and altered mentation (AUROC 0.80) (14). Furthermore, a model based only on administrative variables yielded an AUROC 0.81 (41) despite the inaccuracies inherent in such data (42).

Our proposed performance threshold for AUROC implies that organ failure scores, such as the sequential organ failure assessment (SOFA) and the multiple organ dysfunction score (MODS), generally fail to provide acceptable discriminant accuracy (43,44) to mitigate bias in outcome comparisons used to make inferences regarding quality of care.  Outdated versions of severity scoring systems, such as the mortality probability model (MPM) and APACHE II, may achieve discriminant accuracy in the marginal range, with AUROCs of 0.80-0.84 (3,14,45). Well-designed contemporary severity scoring systems, such as APACHE IV, MPM-III, the simplified acute physiology score (SAPS-3), the Veterans Affairs intensive care unit risk adjustment model (1,3,5,9,15,46,47) and several newer machine-learning models (48,49) generally achieve AUROCs ranging from 0.84-0.89 when applied to general patient populations for which they were designed and validated.

Conclusions

Our study suggests that the explained variance and discriminant accuracy of general severity adjustment scoring systems like APACHE might be significantly reduced when they are used to predict outcomes in specific subgroups of ICU patients, and therefore caution should be exercised in making inferences regarding quality of care based on these predictions. Further studies are needed to establish absolute performance criteria for severity scoring systems.

References 

1. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34:1297-1310. [CrossRef] [PubMed]
2. Zimmerman JE, Kramer AA, McNair DS, Malila FM, Shaffer VL. Intensive care unit length of stay: Benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med. 2006;34:2517-29. [CrossRef] [PubMed]
3. Breslow MJ, Badawi O. Severity scoring in the critically ill: Part 1 – Interpretation and accuracy of outcome prediction scoring systems. Chest. 2012;141:245-52. [CrossRef] [PubMed]
4. Kramer A, Zimmerman J, Yi J. Performance of MPMo-III and APACHE IV mortality predictions in a multi-institutional ICU database – implications for national benchmarking. Crit Care Med. 2012;40:1–328 [Abstract]. [CrossRef]
5. Vincent J-L, Moreno R. Clinical review: Scoring systems in the critically ill. Crit Care. 2010;14:207-16. [CrossRef] [PubMed]
6. Chan T, Bleszynski MS, Buczkowski AK. Evaluation of APACHE IV predictive scoring in surgical abdominal sepsis: A retrospective cohort study. J Clin Diag Res. 2016;10:16-8. [CrossRef] [PubMed]
7. Ghorbani M, Ghaem H, Rezaianzadeh A, et al. A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran. F1000 research. 2017;6:2032. [CrossRef] [PubMed]
8. Afessa B, Gajic O, Keegan MT. Severity of illness and organ failure assessment in adult intensive care units. Crit Care Clinics. 2007;23:639-58. [CrossRef] [PubMed]
9. Keegan MT, Gajic O, Afess B. Comparison of APACHE III, APACHE IV, SAPS 3 and MPMoIII and influence of resuscitation status on model performance. Chest. 2012;142:851-8. [CrossRef] [PubMed]
10. Hadorn DC, Keeler EB, Rogers WH, Brook RH. Assessing the performance of mortality prediction models. RAND/UCLA/Harvard Center for Healthcare Financing Policy Research. 1993. ISBN: 0-8330-1335-1.
11. Verburg IWM, de Keizer NF, de Jonge E, Peek N. Comparison of regression methods for modeling intensive care length of stay. PLOS One. 2014;9:1-11. [CrossRef] [PubMed]
12. DeSalvo KB, Fan VS, McDonell MB, Fihn SD. Predicting mortality and healthcare utilization with a single question. HRS: Health Services Research. 2005;40:1234-46. [CrossRef]
13. Austin PC, Styerberg EW. Interpreting the concordance statistic of a logistic regression model: Relation to the variance and odds ratio of a continuous explanatory variable. Med Care. 2013;51:275-84. [CrossRef] [PubMed]
14. Freund Y, Lemachatti N, Krastinova E, Van Laer M, et al. Prognostic accuracy of Sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA. 2017;317:301-8. [CrossRef] [PubMed]
15. Kuzniewicz MW, Vasilevskis EE, Lane R, et al. Variation in ICU risk-adjusted mortality: Impact of methods of assessment and potential confounders. Chest. 2008;133:1319-27. [CrossRef] [PubMed]
16. Khwannimit B. Bhurayanontachai R, Vattanavanit V. Validation of the sepsis severity score compared with updated severity scores in predicting hospital mortality in sepsis patients. Shock. 2017;47:720-5. [CrossRef] [PubMed]
17. Nassar AP, Mocelin AO, Andrade FM, Brauer L. SAPS 3, APACHE IV or GRACE which score to choose for acute coronary syndrome patients in intensive care units? Sao Paulo Med. J. 2013;131:173-8. [CrossRef]
18. Ayazoglu TA. A comparison of APACHE II and APACHE IV scoring systems in predicting outcomes in patients admitted with stroke to an intensive care unit. Anaesth Pain and Intens Care. 2011;15:7-12. Available at: http://www.apicareonline.com/original-article-a-comparison-of-apache-ii-and-apache-iv-scoring-systems-in-predicting-outcome-in-patients-admitted-with-stroke-to-an-intensive-care-unit/ (accessed 12/26/18).
19. Bauer PR, Kashyap R, League SC, Park JG, et al. Diagnostic accuracy and clinical relevance of an inflammatory biomarker panel for sepsis in adult critically ill patients. Diagn Microbiol Infect Dis. 2016;84:175–80. [CrossRef] [PubMed]
20. Kramer AA, Zimmerman JE. Predicting outcomes for cardiac surgery patients after intensive care unit admission. Semin Cardiothorac Vasc Anesth. 2008;12:175-83. [CrossRef] [PubMed]
21. Costa e Silva VT, de Castro I, Liano F, et al. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant. 2011;26:3894-901. [CrossRef] [PubMed]
22. Soares M, Salluh JI. Validation of the SAPS 3 admission prognostic model in patients with cancer in need of intensive care. Intensive Care Med. 2006;32:1839-44. [CrossRef] [PubMed]
23. Oliveira VM, Brauner JS, Rodrigues Filho EIs SAPS 3 better than APACHE II at predicting mortality in critically ill transplant patients? Clinics (Sao Paulo). 2013;68:153-8. [CrossRef]
24. Tsai CW, Lin YF, Wu VC, et al. SAPS 3 at dialysis commencement is predictive of hospital mortality in patients supported by extracorporeal membrane oxygenation and acute dialysis. Eur J Cardiothorac Surg. 2008;34:1158-64. [CrossRef] [PubMed]
25. Doerr F, Badreldin AM, Can F, et al. SAPS 3 is not superior to SAPS 2 in cardiac surgery patients. Scand Cardiovasc J. 2014;48:111-9. [CrossRef] [PubMed]
26. Salciccioli JD, Cristia C, Chase M, et al. Performance of SAPS II and SAPS III scores in post-cardiac arrest. Minerva Anestesiol. 2012;78:1341-7. [PubMed]
27. Badreldin AM, Kroener A, Heldwein MB, et al. Prognostic value of daily cardiac surgery score (CASUS) and its derivatives in cardiac surgery patients. Thorac Cardiovasc Surg. 2010;58:392-7. [CrossRef] [PubMed]
28. Doerr F, Badreldin AMA, Heldwein MC, et al. A comparative study of four intensive care outcome prediction models in cardiac surgery patients. J Cardiothorac Surg. 2011;6:21. [CrossRef] [PubMed]
29. Hekmat K. Prediction of mortality in intensive care unit cardiac surgical patients, European J Cardiothorac Surg. 2010;38:104–9. [CrossRef] [PubMed]
30. Finnerty CC, Ju H, Spratt H, et al. Proteomics improves the prediction of burns mortality: results from regression spline modeling. Clin Transl Sci. 2012;5:243-9. [CrossRef] [PubMed]
31. Mikacenic C, Price BL, Harju-Baker S, et al. Two-Biomarker Model Predicts Mortality in the Critically Ill with Sepsis. Am J Respir Crit Care Med. 2017;196:1004-11. [CrossRef] [PubMed]
32. Cuenca AG, Gentile LF, Lopez MC, et al. Development of a genomic metric that can be rapidly used to predict clinical outcome in severely injured trauma patients. Crit Care Med. 2013;41:1175-85. [CrossRef] [PubMed]
33. Stortz JA, Mira JC, Raymond SL, et al. Benchmarking clinical outcomes and the immunocatabolic phenotype of chronic critical illness after sepsis in surgical intensive care unit patients. J Trauma Acute Care Surg. 2018;84:342-9. [CrossRef] [PubMed]
34. Muehlschlegel JD, Liu KY, Perry TE, et al. Chromosome 9p21 variant predicts mortality after coronary artery bypass graft surgery. Circulation. 2010;122:S60-5. [CrossRef] [PubMed]
35. Perry TE, Muehlschlegel JD, Liu KY, et al. Preoperative C-reactive protein predicts long-term mortality and hospital length of stay after primary, non-emergent coronary artery bypass grafting. Anesthesiology. 2010;112:607-13. [CrossRef] [PubMed]
36. Scicluna BP, van Vught LA, Zwinderman AH, et al. Classification of patients with sepsis according to blood genomic endotype: a prospective cohort study. Lancet Respir Med. 2017;5:816-26. [CrossRef] [PubMed]
37. Sharma NK, Salomao R. Sepsis through the eyes of proteomics: The progress in the last decade. Shock. 2017;47:17-25. [CrossRef] [PubMed]
38. Miajlovic H, Aogáin MM, Collins CJ, et al. Characterization of Escherichia coli bloodstream isolates associated with mortality. J Med Microbiol. 2016;65:71-9. [CrossRef] [PubMed]
39. Vasilevskis EE, Kuzniewicz MW, Brian A. Cason BA. Mortality Probability Model III and Simplified Acute Physiology Score II: Assessing their value in predicting length of stay and comparison to APACHE IV. Chest. 2009;136:89–101. [CrossRef] [PubMed]
40. Submitted Am J Epidemiol.
41. Male, E, Ghamande S, White HD. Superiority of APACHE IV vs. APR-DRG in evaluating ICU mortality based on discharge billing codes. Am J Resp Crit Care Med. 2017;195:A2772 [Abstract].
42. van Walraven C, Bennett C, Forster AJ. Administrative database research infrequently used validated diagnostic or procedural codes. J Clin Epidemiol. 2011;64:1054-9. [CrossRef] [PubMed]
43. Bota, P, Melot C, Ferreira L, et al. The Multiple Organ Dysfunction Score (MODS) versus the Sequential Organ Failure Assessment (SOFA) score in outcome prediction. Inten Care Med. 2002;28:1619-24. [CrossRef] [PubMed]
44. Zygun DA, Laupland KB, Fick GH, et al. Limited ability of SOFA and MOD scores to discriminate outcome: a prospective evaluation in 1,436 patients. Can J Anaesth. 2005;52:302-8. [CrossRef] [PubMed]
45. Ho KM, Williams TA, Harahsheh Y, Higgins TL. Using patient admission characteristics alone to predict mortality of critically ill patients: A comparison of 3 prognostic score. J Crit Care. 2016;31:21-5. [CrossRef] [PubMed]
46. Render ML, Deddens J, Freyberg R, Almenoff P, et al. Veterans Affairs intensive care unit risk adjustment model: validation, updating, recalibration. Crit Care Med. 2008;36:1031-42. [CrossRef] [PubMed]
47. Lipshutz AKM, Feiner JR, Grimes B, Gropper MA. Predicting mortality in the intensive care unit: a comparison of the University Health Consortium expected probability of mortality and the Mortality Prediction Model III. Intensive Care. 2016;4:35. [CrossRef] [PubMed]
48. Kim S, Kim W, Park RW. A comparison of intensive care unit mortality prediction models through the use of data mining techniques. Healthcare Inform Res. 2011;17:232-43. [CrossRef] [PubMed]
49. Pirracchio R, Petersen ML, Carone M, Rigon MR, Chevret S, van der Laan MJ. Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study. Lancet Respir Med. 2015;3:42–52. [CrossRef] [PubMed]

Acknowledgements

We would like to acknowledge the work of Maria Calleja and Banner Health Clinical Performance Analytics in providing the data used in our analysis.

Author’s contributions

Conception and design: RAR, RDG, KSR, MF, SCC

Data collection: RAR

Statistical analysis: RDG, RAR

Interpretation: RAR, RDG, KSR, MF, SCC

Writing the manuscript: RAR, RDG, KSR, MF, SCC

Guarantor taking full responsibility for integrity of the study: RAR

The authors have no conflicts of interest to report and there was no direct funding for this project.

Abbreviation List

• 2VSS: two-variable scoring system
• APACHE: Acute Physiology and Chronic Health Evaluation
• APS: acute physiology score
• AMI: acute myocardial infarction
• AUROC: area under the receiver operating curve
• CABG: coronary artery bypass grafting
• CASUS: cardiac surgery score
• GI: gastrointestinal
• ICU: intensive care unit
• LOS: length of stay
• MODS: multiple organ dysfunction score
• MPM: mortality probability model
• RAND (corporation): research and development
• R²: Pearson’s coefficient of determination
• SAPS: simplified acute physiology score
• SOFA: sequential organ failure assessment

Cite as: Raschke RA, Gerkin RD, Ramos KS, Fallon M, Curry SC. The explained variance and discriminant accuracy of APACHE IVa severity scoring in specific subgroups of ICU patients. Southwest J Pulm Crit Care. 2018;17:153-64. doi: https://doi.org/10.13175/swjpcc108-18 PDF

Clinical Performance of an Interactive Clinical Decision Support System for Assessment of Plasma Lactate in Hospitalized Patients with Organ Dysfunction

Robert A. Raschke, MD MS

Hargobind Khurana, MD 

Huw Owen-Reece, MBBS 

Robert H. Groves Jr, MD

Steven C. Curry, MD

Mary Martin, PharmD

Brenda Stoffer, RN BSN

Banner University Medical Center Phoenix

Phoenix, AZ USA

Abstract

Purpose: Elevated plasma lactate concentration can be a useful measure of tissue hypo-perfusion in acutely deteriorating patients, focusing attention on the need for urgent resuscitation. But lactate is not always assessed in a timely fashion in patients who have deteriorating vital signs. We hypothesized that an electronic medical record (EMR)-based decision support system could interact with clinicians to prompt assessment of plasma lactate in appropriate clinical situations in order to risk stratify a population of inpatients and identify those who are acutely deteriorating in real-time.

Methods: All adult patients admitted to our hospital over a three month period were monitored by an EMR-based lactate decision support system (lactate DSS) programmed to detect patients exhibiting acute organ dysfunction and engage the clinician in the decision to order a plasma lactate concentration. Inpatient mortality was determined for the five risk categories that this system generated, and chart review was performed on a high-risk subgroup to describe the spectrum of bedside events that triggered the system logic.

Results: The lactate DSS segregated inpatients into five strata with mortality rates of 0.8% (95%CI:0.6-1.0%); 2.7% (95%CI:1.0-4.4%); 7.9% (95%CI: 6.0-10.1%), 13.0% (95%CI: 9.0-17.8%) and 42.1% (95%CI: 32.0-52.4%), achieving a discriminant accuracy of 80% (95%CI:76-84%) by AUROC for predicting inpatient mortality. Classification into the two highest risk strata had a positive predictive value for detecting acute life-threatening clinical events of 54% (95%CI: 41.5-66.5%).

Conclusions: Our lactate decision support system is different than previously-described computerized “early warning systems”, because it engages the clinician in decision-making and incorporates clinical judgment in risk stratification. Our system has favorable operating characteristics for the prediction of inpatient mortality and real-time detection of acute life-threatening deterioration.

Introduction

Over 700,000 deaths occur annually in U.S. hospitals (1). Sepsis accounts directly for 37% and indirectly for 56% of these deaths (2). Other common causes of inpatient mortality such as acute hemorrhage and venous thromboembolism (3) share certain early clinical findings with sepsis, in that they may present with deterioration of vital signs and biochemical variables before life-threatening manifestations become obvious (4). Recognition of these findings provides an opportunity for early intervention, which has been shown to improve mortality (5,6). Studies have shown that failure to rapidly recognize acute clinical deterioration is one of the most common root causes of preventable inpatient mortality (4,8).

Early warning systems (EWSs) are a type of clinical decision support system (CDSS) utilized to provide surveillance of hospitalized patients in order to alert clinicians when a patient has findings associated with acute deterioration (19). These typically monitor for abnormal vital signs or laboratory evidence of organ dysfunction, but have included many other types of clinical and laboratory variables (20-23). Modern EWSs utilize logistic regression to weight up to 36 different independent variables and yield highly stratified risk scores (24-26).

We had previous experience developing a simple EWS that triggered when at least two systemic inflammatory response syndrome (SIRS) criteria plus at least one of 14 acute organ dysfunction (OD) parameters was detected. Although this system references SIRS it was found to be nonspecific for sepsis (27), and was subsequently employed in our healthcare system to identify patients deteriorating in real-time regardless of the cause. Subsequent research showed that our SIRS/OD alert system was triggered during the course of 19% of admissions, and that patients who triggered the alert had an odds ratio of 30.1 (95% CI: 26.1-34.5) for inpatient mortality (28). We hypothesized that this SIRS/OD alert system could be used to identify high risk patients who might be further risk-stratified by obtaining a plasma lactate concentration.

Elevated plasma lactate concentration is a particularly useful biochemical marker of acute decompensation. Hyperlactemia is pathophysiologically associated with acute tissue hypoperfusion, and clinically associated with organ dysfunction and mortality (7-11). Hyperlactemia is also associated with the need for urgent clinical interventions such as transfusion and urgent surgery in trauma patients (13,14), and resuscitation of medical patients with sepsis or other life-threatening illnesses (5,15). Lactate assessment is integral to the definition of sepsis (7,16), and an essential component of the Surviving Sepsis Campaign sepsis resuscitation bundle (6). Lactate assessment is integral to achieving sepsis bundle compliance as defined by the Centers for Medicare and Medicaid Services (CMS), which has mandated participating hospitals to report as a measure of quality of care. However, lactate is only ordered about half the time that it ought to be in patients with severe sepsis and septic shock (17,18). To our knowledge, only one previously reported EWS incorporates lactate assessment (29), but this system passively utilized lactate concentration results obtained on admission from the emergency room and was not used for surveillance during hospitalization.

We sought to use our SIRS/OD alert system to actively trigger lactate assessment to identify patients suffering from sepsis or any other life-threatening disease process requiring immediate intervention during hospitalization. We hypothesized that the resulting “lactate decision support system” (lactate DSS) would provide inpatient mortality risk stratification with high discriminant accuracy, and detect acute life-threatening events with high positive predictive value compared to contemporary EWSs.

A lactate DSS with these favorable characteristics could theoretically be used to guide emergent interventions in an effort to save lives, although it was not our aim at this time to perform an interventional trial. The specific aims of this study were to pilot an interactive lactate DSS in our healthcare system, and to calculate its discriminant accuracy for mortality risk stratification, and its positive predictive value as a real-time early warning system.

Methods

We prospectively studied a cohort of all adult inpatients admitted to Banner-University Medical Center - Phoenix, a 650-bed academic hospital in Phoenix Arizona, during the first quarter of 2014. Our research was part of an ongoing system-level patient safety project and was approved by our Institutional Review Board.

The decision support logic was developed at Banner Health using Discern Expert^® (Cerner Corporation, North Kansas City MO, USA). The lactate decision support system (lactate DSS) monitored each patient in our EMR for vital signs and laboratory results consistent with SIRS and organ dysfunction, using criteria derived from the standard definition of sepsis (5-7) (Table 1).

Table 1. Lactate DSS trigger logic

If criteria for SIRS and organ dysfunction overlapped in any eight-hour window, the lactate DSS was triggered to respond. An electronic notification was generated to the patient’s nurse and physician alerting them to the possibility of acute clinical deterioration suggested by SIRS and organ dysfunction, and recommending evaluation and resuscitation if appropriate. Decision support included automatic generation of an order for a STAT plasma lactate if one was not previously ordered by the clinician, interactively prompting the clinician to cancel it if they felt it was unnecessary.

Adult admissions during the three-month study period and subsequent inpatient mortality were enumerated using our hospital’s general financial database: MedSeries4^® (Siemens Corporation, Washington DC). Although some patients triggered the lactate DSS multiple times over the course of their hospital stay, only the first trigger event was included in our analysis.

Inpatient mortality rates with ninety-five percent confidence intervals were calculated for each of five subgroups: 1) patients who did not exhibit SIRS and organ dysfunction during their hospitalization and therefore did not trigger a lactate DSS response; 2) patients who triggered a lactate DSS response, for whom a DSS-generated lactate order was cancelled by their clinician; 3) patients who triggered the lactate DSS and had a lactate concentration <2.2 mmol/L (normal for our laboratory); 4) patients who triggered the lactate DSS and had an elevated lactate of 2.2-3.9 mmol/L; and 5) patients who triggered the lactate DSS and had a highly elevated lactate >4.0 mmol/L.

It was our hypothesis that mortality in patients who triggered the lactate DSS logic would be equivalent whether the clinician chose not to cancel a DSS-generated lactate order, or the clinician had already entered a lactate order themselves. Therefore, we classified patients into the subgroups above regardless of whether their lactate order was DSS-generated or entered independently by the clinician. In order to confirm the validity of this hypothesis, the mortality rate of all patients with any lactate concentration result (the sum of groups 3, 4 and 5 above), and mortality rates within each lactate concentration strata, were separately analyzed to determine if mortality depended on the method of lactate order entry.

Stratified likelihood ratios and the area under the receiver operating curve (AUROC) generated using the five subgroups described above were calculated to determine the discriminant accuracy of the lactate DSS for the outcome of inpatient mortality.

A subgroup analysis was performed of all study patients with an elevated lactate >2.2 mmol/L (above the upper limit of normal range at our laboratory) detected by a DSS-generated lactate order during the first six weeks of the study. These patients’ charts were reviewed in order to characterize the acute clinical events that triggered a lactate DSS response in this subgroup of patients. A physician researcher reviewed progress notes, laboratory and microbiology results at the time of system activation, and for 72 hours afterwards to make this determination. Patient were determined to be suffering an acute life-threatening clinical event if a new-onset or rapidly-progressive disease process was present at the time the lactate DSS was triggered that required emergent treatment with any one of the following: >1 L intravenous fluid resuscitation, vasopressor infusion, >2 units of packed red blood cell transfusion, endotracheal intubation, advanced cardiac life support, or emergent surgical intervention. Minor clinical events included any diagnosis that required initiation of treatment not included in the definition of acute life threatening clinical events above. False alerts were said to have occurred when no evidence was found that the patient was clinically deteriorating in temporal relationship to lactate DSS activation, or within 72 hours. The positive predictive value of the system was calculated for the real-time detection of acute life-threatening clinical events. Microsoft Research and VassarStats^® on-line statistical software were used for statistical calculations.

Results

8,867 adult patients were admitted during our three-month study period. One hundred and ninety-six of 8867 patients (2.2% 95%CI: 1.9-2.5%) died while in the hospital. Seventy percent (138/196) of these inpatient deaths occurred in the 16% (1400/8867) of patients who triggered a lactate DSS response.

Four hundred seventy-nine of 1400 patients who triggered the lactate DSS already had a clinician-ordered lactate. A DSS-generated order for plasma lactate was entered for the remaining 921 patients, but clinicians cancelled 337 of these. DSS-generated lactate orders were resulted for the remaining 584 patients. These patients were merged with 479 patients who had clinician –ordered lactates for the purposes of further analysis after confirmation that mortality did not depend on how the lactate was ordered (Figure 1).

Figure 1. Stratification of inpatients into five subgroups by the lactate DSS.

Patients who did not trigger the lactate DSS logic (n=7467) had a mortality rate of 0.78% (95%CI: 0.58-0.98). Patients who triggered the lactate DSS and for whom a DSS-generated lactate order was cancelled by the clinician (n=337) had mortality of 2.7% (95%CI: 1.0-4.4%). Patients who triggered the lactate DSS and had a lactate concentration in the normal range (< 2.2 mmol/L; n=721) had mortality of 7.9% (95%CI: 6.0-10.1%), and those with elevated lactates of 2.2-3.9 and >4.0 mmol/L (n=247 and n=95) had mortality rates of 13.0% (95%CI: 9.0-17.8%) and 42.1% (95%CI: 32.0-52.4%) respectively (Figure 2).

Figure 2. Inpatient mortality rates (Y-axis: Percent mortality) with 95% confidence intervals for five subgroups of patients stratified by lactate DSS.

The mortality of patients who triggered a lactate DSS response and for whom a lactate concentration was resulted did not depend on whether the order was DSS-generated or entered by the clinician (13.0% versus 12.1% (P=0.71)). Clinician-entered lactate orders were closely temporally related to the onset of organ dysfunction, preceding lactate DSS triggering by < six hours in 52%, <12 hours in 64%, and <24 hours in 75% of cases. Likelihood ratios for mortality in subgroups of patients with lactates <2.2, 2.2-3.9, and >4.0 mmol/L were 6.1 (95%CI: 5.4-6.9), 11.8 (95%CI: 9.5-14.7), and 32.4 (95%CI: 22.0-47.1) respectively.

Five-strata of mortality risk generated by the lactate DSS yielded an AUROC of 0.80 (95% CI: 0.76-0.84) (Figure 3).

Figure 3. Receiver-operating characteristic curve for mortality risk stratification by the lactate DSS.  

Focused chart review was performed on 61 patients who had elevated lactate (>2.2 mmol/L) detected by a DSS-generated lactate order. Thirty-three (54%) were experiencing acute life-threatening clinical events at the time the lactate DSS was triggered. These included 18 episodes of sepsis. Sepsis was due to pneumonia in nine patients, catheter-associated blood stream infection, bowel perforation, cellulitis, ascending cholangitis, endocarditis, liver abscess, cholecystitis, perianal abscess, or an unidentified source. Other acute life threatening clinical events included five cases of acute gastrointestinal hemorrhage, three of acute respiratory failure, and one each of post-operative bleeding, cardiogenic shock, acute liver failure, retroperitoneal bleeding, acute myocardial infarction, subdural hematoma, and cerebral dural sinus thrombosis. Twenty-one (64%) of these events occurred outside the intensive care unit. The positive predictive value of the detection of SIRS, organ dysfunction and elevated lactate by the lactate DSS for acute life-threatening clinical events was 54% (95%CI: 41.5-66.5%).

Ten minor clinical events included anemia, atrial fibrillation, post-op third spacing, transient mild hypotension associated with end stage liver disease, sedation related to narcotics, and dialysis disequilibrium. There were 18 false alerts among patients with SIRS, organ dysfunction and elevated lactate detected by the system. (18/61=29%).

Discussion

Our lactate DSS effectively segregated a population of adult inpatients into five subgroups with increasing inpatient mortality. Clinician engagement was critically important in achieving this result. About a quarter (337/1400) of patients who triggered the lactate DSS (simultaneously exhibited SIRS and organ dysfunction) were doing well enough in their clinician’s opinion that the DSS-generated lactate order was cancelled. Clinicians exercised good judgment in this regard, identifying a subgroup of patients with inpatient mortality rate not significantly higher than the overall mortality of all patients admitted during the study. This supports our decision to incorporate clinician judgment in our risk stratification method.

Approximately half of patients (721/1400) who triggered the lactate DSS turned out to have a normal lactate concentration, yet suffered inpatient mortality ten-times higher than patients who did not trigger the system. This likely represents the independent association between SIRS and organ dysfunction with the risk for mortality (27, 31,32).

One hundred twenty-nine patients over 3 months (14.5 per 1000 patient admissions) triggered the lactate DSS and were found to have an elevated lactate concentration because of a DSS-generated lactate order. These patients had >50% probability of experiencing an acute life-threatening clinical event at the time the lactate DSS was triggered, and subsequently suffered 50% inpatient mortality.

Our lactate DSS is consistent with the new definition of sepsis because it uses organ dysfunction in addition to SIRS criteria (7). As stated in the new definition of sepsis, “Nonspecific SIRS criteria such as pyrexia or neutrophilia will continue to aid in the general diagnosis of infection” (7). Although these criteria are nonspecific, they appear to be relatively sensitive for sepsis (7,27). Our lactate DSS has excellent discriminant accuracy for predicting inpatient mortality (AUROC=0.80). It is comparable to other criteria such SOFA (AUROC = 0.74) and the Logistic Organ Dysfunction System (AUROC=0.75).The five strata into which it segregates patients could further translate into a decision support-guided treatment protocol, directing appropriate real-time interventions such as those proposed in Table 2.

Table 2. Proposed stratified clinical response to lactate DSS.

* Our data indicate that RRT activation would occur about twice a week at our hospital.

Our lactate DSS is different than EWSs because it specifically prompts assessment of plasma lactate in patients exhibiting SIRS and organ dysfunction, rather than simply generating a warning. But a discussion of the operating characteristics of previously reported EWSs is useful for purposes of comparison. A review of 33 EWSs has reported AUROCs ranging from 0.66-0.78 (19). Several more recent EWSs reported AUROCs of 0.81-0.88 (23,24,26,33), but AUROC comparisons are confounded by lack of consensus regarding which clinical outcome to analyze. Authors have variously chosen 24-hour mortality, ICU transfer, and cardiac arrest, among other outcomes (20,23,24). Many EWSs yield highly stratified results, which may increase the AUROC by adding detail to the shape of the ROC curve, but this will not improve clinical discrimination unless each resulting strata has a distinct clinical response. If a EWS is simply used to activate a rapid response team (RRT), the clinically-achievable discriminant accuracy is best described by a polygonal AUROC derived from a single cutoff with two resulting strata (activate the RRT, or do not activate the RRT). This two-strata AUROC will invariably be lower than the highly stratified AUROC that many authors report (23,24,26,33). Our AUROC analysis is based on 5 strata, each of which could reasonably trigger a distinct clinical response (Table 2).

Our lactate decision support system has a positive predictive value (PPV) for acute life-threatening clinical events that is superior to that of our previous “sepsis alert” (27) and to those reported in several reviews of EWSs. One review of 39 EWSs reported PPVs ranging from 13.5-26.1% (34), and another review of 25 systems reported a median PPV of 36.7% with interquartile range 29.3-43.8% (34). PPV was not reported for several of the most elegant and well-studied EWSs (22,23,25,32). From the perspective of bedside clinicians and rapid response team members, the efficiency of an alert system is strongly influenced by the PPV, because a poor PPV translates to frequent false alerts. The PPV is of particularly concern when the pretest probability of the outcome of interest is low, as in the case of inpatient mortality (2% at our hospital). Bayes theory indicates that a test with relatively good AUROC will have a poor PPV if the pretest probability is low enough.

Our study has several limitations. Our sample size is small compared to many contemporary EWS studies. We did not have the resources to perform focused chart reviews on all study patients and therefore had to limit individual case analysis to a subgroup of study patients. Our simple treatment of vital sign abnormalities as markers of SIRS is not as elaborate as in many EWSs. Our study is only hypothesis-generating, whereas several EWSs are well validated (25,32). We cannot provide data on how our alert might change bedside interventions by clinicians. To our knowledge, no study to date has proven that using a computerized decision support system or EWS to trigger rapid clinical intervention actually improves patient outcomes.

Conclusions

We developed an automated decision-support system that prompts assessment of plasma lactate concentration in patients exhibiting SIRS and organ dysfunction. Our lactate decision support system is different than previously-described EWSs because it engages the clinician in decision-making and incorporates clinical judgment into risk stratification. This system has favorable operating characteristics for the prediction of inpatient mortality and for detecting acute life-threatening events in real time. We have proposed a stratified clinical response based on classification of patients into five subgroups by this system that requires further testing, but our current study was not designed to demonstrate a benefit on clinical outcomes. Our lactate DSS has the potential to improve sepsis bundle compliance by helping clinicians appropriately order lactate concentrations in patients deteriorating due to the onset of sepsis – a hypothesis we are currently investigating. It also has potential for easy generalizability, particularly to other healthcare systems that share the same EMR as ours, but requires further refinement and validation.

Author Contributions

All authors were involved in conceptualization, design and implementation of the decision support system described in this manuscript, and in preparation of the manuscript, and all approve of the content of the manuscript and vouch for the validity of the data. We list below additional contributions from several of the authors:

RAR: data analysis and interpretation, main author of initial draft of the manuscript.

HOW: data analysis and interpretation, contribution to discussion/conclusions

HK: directly in charge of design and pilot implementation team for the decision support system, data interpretation, contribution to discussion, conclusions

RHG: data interpretation, contribution to discussion, conclusions

SCC: data analysis and interpretation, contribution to discussion, conclusions. Manuscript editing.

MM: data collection and analysis

BS: data collection and analysis

References

1. Center for Disease Control. Trends in inpatient hospital deaths: National Hospital Discharge Survey: 2000:2010. NCHS Data Brief 118; 2013. [PubMed]
2. Liu V, Escobar G, Greene JD, Soule J, et al. Hospital deaths in patients with sepsis from two independent cohorts. JAMA. 2014;312:90-92. [CrossRef] [PubMed]
3. Nichols L, Chew B. Causes of sudden unexpected death of adult hospital patients. J Hosp Med. 2012;7:706-8. [CrossRef] [PubMed]
4. McGloin H, Adam SK, Singer M. Unexpected deaths and referrals to intensive care of patients on general wards. Are some cases potentially avoidable? J R Coll Physicians Lond. 1999;33:255-9. [PubMed]
5. Rivers E, Nguyen B, Havstad S, Ressler J, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. NEJM. 2001;345:1368-77. [CrossRef] [PubMed]
6. Dellinger RP, Levy MM, Rhodes A, Annane D, et al. Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41:580. [CrossRef] [PubMed]
7. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016 Feb 23;315(8):801-10. [CrossRef] [PubMed]
8. National Patient Safety Agency. Safer care for the acutely ill patient: learning from serious incidents. 2007;Report # PSO/5. Available online at: http://www.nrls.npsa.nhs.uk/resources/?EntryId45=59828 (accessed 5/9/17).
9. Gultepe E, Green JP, Nguyen H, Adams J, et al. From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system. J Am Med Inform Assoc. 2014;21:315-325. [CrossRef] [PubMed]
10. Jansen TC, van Bommel J, Woodward R, Mulder PG, Bakker J. Association between blood lactate levels, sequential organ failure assessment sub-scores, and 28-day mortality during early and late intensive care unit stay: a retrospective observational study. Crit Care Med. 2009;37:2369-74. [CrossRef] [PubMed]
11. Bakker J, Gris P, Coffernils M, Kahn RJ, Vincent JL. Serial blood lactate levels can predict the development of multiple organ failure following septic shock. Am J Surg. 1996;171:221-6. [CrossRef] [PubMed]
12. Jansen TC, van Bommel J, Bakker J. Blood lactate monitoring in critically ill patients: a systematic health technology assessment. Crit Care Med. 2009;37:2827-39. [CrossRef] [PubMed]
13. Guyette F, Suffoletto B, Castillo JL, Quintero J, Callaway C, Puyana JC. Prehospital serum lactate as a predictor of outcomes in trauma patients: A retrospective observational study. J Trauma. 2011;70:782–6. [CrossRef] [PubMed]
14. Vandromme MJ, Griffin RL, Weinberg JA, Rue LW 3rd, Kerby JD. Lactate is a better predictor than systolic blood pressure for determining blood requirement and mortality: Could prehospital measures improve trauma triage? J Am Coll Surg. 2010;210:861–9. [CrossRef] [PubMed]
15. Jansen TC, van Bommel J, Schoonderbeek FJ, Sleeswijk Visser SL, et al. Early lactate-guided therapy in intensive care unit patients: a multicenter, open-label, randomized controlled trial. Am J Respir Crit Care Med. 2010;182:752-61. [CrossRef] [PubMed]
16. Levy MM, Fink MP, Marshall JC, Abraham E, et al. International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250. [CrossRef] [PubMed]
17. Gao R, Melody T, Daniels DF, Giles S and Fox S. The impact of compliance with 6-hour and 24-hour sepsis bundles on hospital mortality in patients with severe sepsis: a prospective observational study. Crit Care. 2005;9:R764–R770. [CrossRef] [PubMed]
18. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med. 2010; 36: 222–31. [CrossRef] [PubMed]
19. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted "track and trigger" systems. Resuscitation. 2008;77:170-9. [CrossRef] [PubMed]
20. Hodgetts TJ, Kenward G, Vlachonikolis IG, Payne S, Castle N. The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team. Resuscitation. 2002;54:125-31. [CrossRef] [PubMed]
21. Kho A, Rotz D, Alrahi K, Cardenas W, et al. Utility of commonly captured data from an HER to identify hospitalized patients at risk for clinical deterioration. AMIA 2007 symposium proceedings. 404-8.[CrossRef]
22. Howell MD, Donnino M, Clardy P, Talmor D, Shapiro NI. Occult hypoperfusion and mortality in patients with suspected infection. Intensive Care Med. 2007;33:1892-9. [CrossRef] [PubMed]
23. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, et al. Early detection of impending physiological deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7:388-95. [CrossRef] [PubMed]
24. Prytherch DR, Smith GB, Schmidt P, Featherstone PI. ViEWS – towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81:932-7. [CrossRef] [PubMed]
25. Bailey TC, Yixin C, Mao Y, Lu C, et al. A trial of a real-time alert for clinical deterioration in patients hospitalized on general medical wards. J Hosp Med. 2013;8:236-42. [CrossRef] [PubMed]
26. Churpek MM, Yuen TC, Winslow C, Robicsek AA, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Resp Crit Care Med. 2014;190:649-55. [CrossRef] [PubMed]
27. Raschke RA, Owen-Reece H, Khurana H, Groves RH Jr, et al. Clinical performance of an automated systemic inflammatory response syndrome (SIRS)/organ dysfunction alert: a system-based patient safety project. Southwest J Pulm Crit Care. 2014;9:223-9. [CrossRef]
28. Khurana, H, Groves RH, Simons MP, Martin M, Stoffer B, et al. Real-time automated continuous sampling of electronic medical records predicts hospital mortality. Am J Med. 2016 Jul;129(7):688-698.e2. [CrossRef] [PubMed]
29. Jo S, Lee JB, Jin YH, Jeong TO, et al. Modified early warning score with rapid lactate level in critically ill medical patients: the ViEWS-L score. Emerg Med J. 2013;30:123-9. [CrossRef] [PubMed]
30. Rangel-Frausto MS, Pittet, D, Costigan M, et al. The natural history of the Systemic Inflammatory Response Syndrome (SIRS): A prospective study. JAMA. 1995;273:117-123. [CrossRef] [PubMed]
31. Matthew M, Churpek F, Zadravecz J, et al. Incidence and prognostic value of the Systemic Inflammatory Response Syndrome and organ dysfunctions in ward patients. Am J Resp Crit Care Med. 2015;192:958-64. [CrossRef] [PubMed]
32. Pittet D, Range-Frausto S, Tarara LN, Lin N, et al. Systemic inflammatory response syndrome, sepsis, severe sepsis and septic shock: incidence, morbidities and outcomes in surgical ICU patients. Intensive Care Med. 1995;21:302-9. [CrossRef] [PubMed]
33. Kellett J, Kim A. Validation of an abbreviated Vitalpac early warning score (ViEWS) in 75,419 consecutive admission to a Canadian regional hospital. Resuscitation. 2012;83:297-302. [CrossRef] [PubMed]
34. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, et al. A review and performance evaluation of single-parameter "track and trigger" systems. Resuscitation. 2008;79:11-21. [CrossRef] [PubMed]
35. Gao H, McDonnell A, Harrison DA, Moore T, et al. Systemic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007;33:667-79. [CrossRef] [PubMed]

Cite as: Raschke RA, Khurana H, Owen-Reece H, Groves RH Jr, Curry SC, Martin M, Stoffer B. Clinical performance of an interactive clinical decision support system for assessment of plasma lactate in hospitalized patients with organ dysfunction. Southwest J Pulm Crit Care. 2017;14:241-52. doi: https://doi.org/10.13175/swjpcc058-17 PDF 

Organ Failure in Acute Pancreatitis and Its Impact on Outcome in Critical Care

Namrata Maheshwari, MD, IDCCM

Arun Kumar, MD 

Zafar A Iqbal, MD

Amit K Mandal, DNB,DTCD

Abhishek Vyas, MBBS

Jai D Wig, MS

Department of Critical Care Medicine and Pulmonology

Fortis Hospital 

Mohali, Punjab, 160062

India 

Abstract

The most important determinant of mortality in acute pancreatitis is organ failure (OF). The aim of this prospective observational study was to determine the incidence of organ failure in acute pancreatitis and its relation with the extent of necrosis and outcome. Sixty-one patients were divided into 3 groups: no organ failure (NOF), transient organ failure (< 48 hrs) (TOF) or persistent organ failure (> 48 hrs) (POF). Of 61 patients, 30 patients had no organ failure (49.1%), while 11 patients (18%) had TOF and 20 patients (32.7%) had POF. The mean age was 46.5 yrs with male predominance. Pulmonary and renal failures were the most common (32%), followed by CVS (cardiovascular system), coagulation system and CNS (central nervous system). Fourteen (46.4%) patients had one or two OF, 17 (56.6%) had more than two OF. There were no deaths in patients with up to two organ failures but a 70% (7) death rate in those with three organ involvement, 80% (4) with four and 100% with five OF. The percentage of pancreatic necrosis was evaluated for its relationship with organ failure. In the NOF group 19 (63.3%) patients had no necrosis, as compared to 11 patients with necrosis in TOF and POF groups (35.4%). Out of 61 patients, 13 patients died. All 13 patients who expired belonged to the POF group (p <.001). Early persisting and deteriorating organ failure had the worst outcomes. There was an increase in mortality with an increasing number of organs involved. The extent of necrosis was directly related with incidence of organ failure.

Introduction

Acute pancreatitis (AP) is characterized by a variable clinical course varying from a mild self-limited disease (80-90%) to a clinically severe acute pancreatitis (SAP) in 10-20% (1-4). Despite advances in knowledge and treatment of AP, the identification of patients with clinically severe disease on admission remains difficult (1) and the mortality in several series continues to be around 20% (2,5). 

The factors responsible for high mortality in patients with SAP are organ failure (OF) and pancreatic necrosis (6,7). The reported incidence of OF in SAP varies from 28-76 % (5,8,9). The occurrence of organ dysfunction and progressive organ failure has a major impact on outcome. Many patients who succumb to AP within the first two weeks of disease onset do so from overwhelming multiorgan failure (10,11). Other studies have also reported that prognosis deteriorated with an increase in number of organs involved (12,13). Banks and Freeman (14) studied the correlation between mortality and organ failure in patients with acute pancreatitis and documented a median mortality of 3% in patients with single organ failure and 47% in patients with multisystem organ failure. Another study documented that the overall mortality (47.8%) correlated with the number of organs failing (6). The definition of multiorgan failure is broad and encompasses transient to persistent or severe multiorgan failure that requires critical care support (15). Patients with persistent organ failure have a higher mortality as compared to patients where organ failure resolves (16). Johnson and Hial (17) showed that patients with OF that resolved within 48 hours(transient) have a low risk of complications and death in comparison to patients who have persistent organ failure(OF persisting for 3 or more days) and have a greater than one in three risk of fatal outcome. Information regarding the prediction of persistent organ failure in patients with acute pancreatitis is not available (18).

One of the factors linked to the development of OF is the extent of pancreatic necrosis. Some workers have found a correlation between the extent of necrosis and OF (19). The question of the relationship between infected necrosis and OF remains unsettled. There is no consistency in the literature on whether organ failure or infected necrosis is the main determinant of severity in acute pancreatitis. The aim of study was to study the occurrence of organ failure in acute pancreatitis and determine the influence of organ failure on mortality in patients with acute pancreatitis.

Materials and Methods

This study was a prospective study under taken during 18 months (December 2011 to May 2013) in the Departments of Gastroenterology, General Surgery and Medical Intensive Care Unit in Fortis Hospital, Mohali, Punjab, a 260 bedded multispecialty tertiary care hospital in Northern India.

The study sample included all consecutive patients diagnosed with acute pancreatitis referred to Gastroenterology or General surgery units fulfilling the inclusion and exclusion criteria. All the patients were assessed for demographic profile and detailed symptom profile. After a detailed clinical examination relevant investigations were repeated as and when required. Patients were monitored for the presence and severity of organ failure every day during the first week, subsequent local complications, subsequent episodes of sepsis, and death or other outcomes during the same hospital admission.

Organ failure was defined as per modified multiple organ failure score (MMOFS). Transient organ failure was defined as organ failure present for less than 48 hours, and persistent organ failure was recorded when organ failure was present for more than 48 hours, where day 0 was the day of entry to the study and day one started at 8.00am on the day after entry. The course in hospital and final outcome was recorded. Cross tabulations were made with outcome, in particular with mortality.

Statistical Analysis. The data are presented as mean ± SD or median and interquartile range, as appropriate. The Mann- Whitney U-test was used for statistical analysis of skewed continuous variables and ordered categorical variables. For normally distributed data The t-test was applied. Pearson χ2 test or Fisher’s exact test was used for analysis of categorical variables with two categories. A p value of <0.05 was considered to indicate statistical significance. All calculations were performed using SPSS® version 15 (Statistical Packages for the Social Sciences, Chicago, IL).

Results

The study was comprised of 61 patients who met the inclusion criteria with diagnosis of acute pancreatitis. The study group was further divided as per organ failure into three groups:

• No organ failure (NOF)
• Transient organ failure ( < 48 hrs) (TOF)
• Persistent organ failure ( > 48 hrs) (POF)

Demographic Distribution. The mean age of the patients was 46.5 years. The majority of patients were in the age group of 30-50 years. In this study the youngest patient was 17 years old and oldest was 87 years old (Figure 1).  

Figure 1. Age distribution with increased number of organs involvement.

The male to female ratio was found to be 2.4:1 (Figure 2).

Figure 2. Sex distribution.

Male predominance was found in all groups (53.3 %, 81.8%, and 90 % in the no organ, transient and persistent organ failure group respectively).

Comorbid Conditions. A majority of the patients (38) in our study group had no associated comorbid conditions  while 23 patients (37.8%) had a previous comorbid condition. Hypertension was the most common comorbid condition, seen in almost 31 % of the patients at the time of admission. Type 2 diabetes was the second most common condition noted in 24.6%, followed by hypothyroidism (4.9%), asthma, depression, cardiomyopathy and Guillain-Barré syndrome in 1.6% each (Table 1).  

Table 1. Comorbid conditions associated in our study group.

 

We could not find any association between co morbidities and mortality as 9 (62.9%) deaths occurred in the no comorbidity group as compared to 4 (30.8%) deaths in the co morbidities group (p=0.880).

Etiology. The most common etiologies of pancreatitis in our study group were alcohol and gall stones (n=24, 39% each) (Table 2).

Table 2. Etiology of acute pancreatitis.

Other causes were idiopathic (n=10, 17%), hypertriglyceridemia (n=2, 3%) and pancreatic divisum (n=1, 2%).

Percentage of Necrosis and Organ Failure. The percentage of necrosis on radiological imaging (in 46 patients) was evaluated for its relationship with organ failure. In the NOF group 19 (63.3%) patients had no necrosis (0%), 4 (13.3%) patients had <30% necrosis, 1 (3.3%) had 30-50% and 4 (13.3%) had >50% necrosis (Figure 3).

Figure 3. Relation between organ failure and pancreatic necrosis.

In the TOF group, 4 (36.4%) patients revealed no necrosis on contrast-enhanced computerized tomography (CECT) of the abdomen, <30% necrosis in 2 (18.2%) patients, 30-50% necrosis in 3 (27.3%) and >50% in 1 (9.1%) patient (Figure 3).

In POF group no necrosis was detected in 3 (15%) patients, <30 % in 2 (10%), 30-50% in 1 (5%) and >50% in 2 (10%) patients. The relationship between the amount of necrosis was directly related with incidence of organ failure and this correlation was found to be statistically significant (Figure 3).

MMOFS and Mortality. We divided our study in 3 groups, no organ failure, transient (<48 hrs) and persistent (>48 hrs) organ failure to understand the nature and dynamics of organ failure. Groups were further divided in early onset (<7days), late onset (>7days). Organ failure was calculated by the Modified multiorgan failure score (MMOFS). Daily MMOFS was calculated in all patients up to 7 days. MMOFS difference was calculated by MMOFS 7 (MMOFS at day 7) – MMOFS 1 (at the time of admission). On the basis of MMOFS difference groups were further divided into same (if difference was 0), improving (if deference was negative value), or deteriorating (if deference was a positive value) groups (Figure 4).

Figure 4. Comparison of outcome with MMOFS difference.

MMOFS difference was found to be highly significantly (ANOVA, p<0.001 each) correlated with organ failures and outcome. In our study no deaths occurred in the transient OF groups (early transient, late transient and transient deteriorating). We attributed this to the dynamics that transient OF could resolve with treatment and had a better outcome than persistent OF. Among the 13 deaths reported in our study, 46.2 % were in the early (<7 days) OF group compared to the late (>7 days) OF group (20%).

Organ Involvement. Pulmonary and renal failures were the most common organ involvements noted among our study group (32% each). This was followed by cardiovascular system (22%), coagulation system (8%) and central nervous system (6%) (Figure 5).

Figure 5. Organ failure by system.

Organ involvement and mortality. Fourteen (46.4%) patients had one or two OF and 17 (56.6%) had more than two OF (table 3). Comparison of the number of organ failures to mortality was statically significant (p<0.001) (Figure 6).

Figure 6. Outcome in patients with increasing organ involvement.

We found that there was an increase in incidence of mortality with an increase in the number of organs involved. There were no deaths in patients with up to two organ failures; it increased with increasing number of organs involved (Table 3).

Table 3. Organ failure and mortality.

The mortality rate was 70% (n=7) with three organ involvement, 80 % (n=4) with four and 100% with five OF.

Discussion

Severe acute pancreatitis is a systemic disease and characterized by acute onset and rapid progression, with a high incidence of complications and serious morbidity (20). An international multidisciplinary classification of acute pancreatitis severity is based on local and systemic determinants of severity. The local determinants relate to presence of pancreatic necrosis, and whether the necrosis is infected or sterile. The systemic determinants relate to whether there is organ failure or not, and if present, whether it is transient or persistent. The presence of both infected pancreatic necrosis and persistent organ failure has a greater impact on severity than either determinant alone. Based on these principles, the severity is classified as mild, moderate, severe or critical (21).The three most common systems involved are renal, lung, and cardiovascular system. Respiratory complications are frequent in acute pancreatitis and respiratory dysfunction is a major component of multiple organ dysfunction syndrome (22,23). In a population based study, 15.05% of patients with AP had a diagnosis of acute renal failure (24).

The present study showed that the difference in age was not significantly different between the groups. There are some studies which showed an association between advancing age as a predictor of organ failure and mortality. Wig et al. (6) studied 161 patients and concluded that age of the patients was a risk factor for multiple organ failure. Li et al. (25) studied 181 patients with SAP and found a correlation of age with OF (<.001). Frey et al²⁶ also showed that the number of complications was positively correlated with the age of patients. Older age and number of complications were strong predictors of organ failure among patients with SAP. Though we recorded a higher incidence of organ failures and mortality in a younger age group of 40-45, the difference was attributed to a small number of patients above 65 years in our study as compared to studies done in the western world.

The bedside index for severity in acute pancreatitis (BISAP) score represents a simple way to identify patients at risk for increased mortality and the development of intermediate markers of severity within 24 hours of presentation. In our series the BISAP score was significantly associated (p<.001) with organ failure as well as survival (p<.001). We found 9 out of 13 deaths in the >3 score group and four deaths at a BISAP score of 2 as compared to zero mortality in the BISAP score 1 and 0 group. Kim et al. (27) also compared BISAP, the serum procalcitonin (PCT), and other multifactorial scoring systems simultaneously, concluded that BISAP is more accurate for predicting the severity of acute pancreatitis than the serum procalcitonin, APACHE-II, Glasgow, and modified CT severity index (MCTSI) scores. Chen et al. (28) evaluated the accuracy of BISAP in predicting the severity and prognosis of acute pancreatitis (AP) in 497 Chinese patients. They conclude that BISAP score is valuable in predicting the severity of AP and prognoses of SAP in Chinese patients.

Contrast enhanced computed tomography (CECT) is considered the gold standard for the diagnosis of pancreatic necrosis and peripancreatic collections. CT assessment correlates with the clinical course of the disease and recognized variables of disease severity. We ordered CECT in all patients on the second or third day after admission rather than at the time of admission. Additional contrast-enhanced CT scans were ordered at intervals during the hospitalization to detect and monitor the course of intra-abdominal complications of acute pancreatitis, such as the development of organized necrosis, pseudocysts, and vascular complications including pseudoaneurysms. In our study CT severity index (CTSI) > 7 at admission did not correlate well with organ failure or mortality (p=NS), although the percentage of necrosis had significant correlation with organ failure. Our results are similar to many studies reported in the literature. Simchuk et al. (29) performed a study on 268 patients with acute pancreatitis. They concluded CTSI > 5 correlated significantly with death. Similar results were also obtained by Leung et al. (30) on 121 patients studied retrospectively, and they concluded that CTSI is superior to Ranson’s score and APACHE II score in predicting outcome in pancreatitis. However a few studies found no association between grade of necrosis and outcome of pancreatitis. Shinzeki et al. (31) did not find any correlation between necrosis evident on CECT at admission and outcome of SAP (p=0.061). Another study by Lankisch et al. (32) also did not find any correlation between necrosis and organ failure.

In our study pulmonary and renal were the most common organ failures observed (32% each). The total number of organ failures at admission was also significantly different in both groups (p=0.001), however none of the organ failures independently proved to be a significant predictor of mortality. MMOFS difference was found to be highly significantly correlated with organ failures and outcome. Among the 13 deaths reported in our study, 46.2 % were in the early (<7 days) OF group compared to the late (>7 days) OF group (20%). Our series also showed comparable results with other studies suggesting that early organ failure is the major predictor of poor outcome MMOFS difference was found to be highly significantly (ANOVA, p<0.001 each) correlated with organ failures and outcome. In our study no deaths occurred in the transient OF groups (early transient, late transient and transient deteriorating). We attributed this to the dynamics that transient OF could resolve with treatment and had a better outcome than persistent OF. Among the 13 deaths reported in our study, 46.2 % were in the early (<7 days) OF group compared to the late (>7 days) OF group (20%). Our series also showed comparable results with other studies suggesting that early organ failure is the major predictor of poor outcome (p=0.002) compared to late organ failure (p=0.400).

We found that early persistent OF had a 66.6% mortality as compared to persistent deteriorating organ failure which also had a very high mortality (72.2%). Very few studies have reported on the dynamics of OF with MMOFS. Johnson et al. (19) in a study of 290 patients with SAP had 116 patients with no OF and 147 patients with OF at the time of admission subdivided those with OF into those with persistent (OF lasting for>48 hours) and transient (OF lasting for<48hours) organ failure. Mortality was 36.3% in persistent and 5% in transient OF group. No patients without OF died.

On analysis of the 13 patients who expired, 4 patients died early (<7 days) and 9 deaths were late (>7 days). OF was the main cause of death in both groups, however all patients with sepsis died later. In the study of Yang et al. (33) the most important and common cause of death for patients with fulminant pancreatitis was multiple organ dysfunction syndrome, which usually was the consequence of systemic inflammation response syndrome in the early stage, and severe infection in the later stage, respectively.

Conclusions

Patients with persistent organ failure have a higher mortality. Early persisting and deteriorating organ failure had the worst outcome of among patients with acute pancreatitis. There was an increase in mortality with increasing number of organs involved. The extent of necrosis was directly related with the incidence of organ failure.

References

1. Bollen TL, Singh VK, Maurer R, Repas K, van Es HW, Banks PA, Mortele KJ. A comparative evaluation of radiologic and clinical scoring system in the early prediction of severity in acute pancreatitis. Am J Gastroenterol. 2012;107:612-9. [CrossRef] [PubMed]
2. Zyromski NJ. Necrotizing pancreatitis in 2010: an unifinished odyssey. Ann Surg. 2010; 251:794-5. [CrossRef] [PubMed]
3. Kylanpaa L, Rakonczay Z, O' Reilly DA. The clinical course of acute pancreatitis and the inflammatory mediators that drive it. Int J inflammation. 2012;2012:360685. [CrossRef] [PubMed]
4. Refinetti RA, Martinez R. Necrotising pancreatitis: update and when to operate. Clinical Medicine and Diagnostics. 2012;2:20-6.
5. Poves Prim I, Fabregat Pous J, García Borobia FJ, Jorba Martí R, Figueras Felip J, Jaurrieta Mas E. Early onset of organ failure is the best predictor of mortality in acute pancreatitis. Rev Esp Enform Dig (Madrid). 2004;96:705-13. [PubMed]
6. Wig JD, Bharathy KG, Kochhar R, Yadav TD, Kudari AK, Doley RP, Gupta V, Babu YR. Correlates of organ failure in severe acute pancreatitis. JOP. 2009;10:271-5. [PubMed]
7. Mole DJ, Olabi B, Robinson V, Garden OJ, Parks RW. Incidence of individual organ dysfunction in fatal acute pancreatitis: analysis of 1024 death records. HPB (Oxford). 2009;11:166-70. [CrossRef] [PubMed]
8. Halonen KI, Pettilä V, Leppäniemi AK, Kemppainen EA, Puolakkainen PA, Haapiainen RK. multiple organ dysfunction associated with severe acute pancreatitis. Crit Care Med. 2002;30:1274-9. [CrossRef] [PubMed]
9. Lytras D, Manes K, Triantopoulou C, Paraskeva C, Delis S, Avgerinos C, Dervenis C. Persistent early organ failure: defining the high-risk group of patients with severe acute pancreatitis? Pancreas. 2008;36:249-54. [CrossRef] [PubMed]
10. Vege SS, Gardner TB, Chari ST, Munukuti P, Pearson RK, Clain JE, Petersen BT, Baron TH, Farnell MB, Sarr MG. Low mortality and high morbidity in severe acute pancreatitis without organ failure: a case for revising the Atlanta classification to include moderately severe acute pancreatitis. Am J Gastroentrol. 2009;104:710-5. [CrossRef] [PubMed]
11. Frey CF, Zhou H, Harvey DJ, White RH. The incidence and case fatality rates of acute biliary, alcoholic and idiopathic pancreatitis in California 1994-2001. Pancreas. 2006;33:336-44. [CrossRef] [PubMed]
12. Buter A, Imrie CW, Carter CR, Evans S, McKay CJ. Dynamic nature of early organ dysfunction determines outcome in acute pancreatitis. Br J Surg. 2002;89:298-302. [CrossRef] [PubMed]
13. Mofidi R, Duff MD, Wigmore SJ, Madhavan KK, Garden OJ, Parks RW. Association between early systemic inflammatory response, severity of multiorgan dysfunction and death in acute pancreatitis. Br J Surg. 2006:93:738-44. [CrossRef] [PubMed]
14. Banks PA, Freeman ML. Practice guidelines in acute pancreatitis. Am J Gastroenterol. 2006;101:2379-2400. [CrossRef] [PubMed]
15. Pandol SJ, Saliya AK, Imrine CW, Banks PA. Acute pancreatitis :Bench to bed side : Gasroentrology. 2007;133:1127-51. [CrossRef] [PubMed]
16. de-Madaria E, Soler-Sala G, Lopez-Font I, Zapater P, Martínez J, Gómez-Escolar L, Sánchez-Fortún C, Sempere L, Pérez-López J, Lluís F, Pérez-Mateo M. Update of Atlanta classification of severity of acute pancreatitis: should a moderate category be included. Pancreatology. 2010;10: 613-9. [CrossRef] [PubMed]
17. Johnson CD, Abu-Hilal M. Persistent organ failure during the first week as a marker of fatal outcome in acute pancreatitis. Gut. 2004;53;1340-4. [CrossRef] [PubMed]
18. Mentula P, Kylanpaa ML, Kemppainen E. Early prediction of organ failure by combined markers in patients with acute pancreatitis. Br J Surg 2005; 92: 68-75. [CrossRef] [PubMed]
19. Garg PK, Madan K, Pande GK, Khanna S, Sathyanarayan G, Bohidar NP, Tandon RK. Association of extent and infection of pancreatic necrosis with organ failure and death in acute necrotizing pancreatitis. Clin Gastroentrol Hepatol. 2005;3:159-66. [CrossRef] [PubMed]
20. Pavlidis P, Crichton S, Lemmich Smith J, Morrison D, Atkinson S, Wyncoll D, Ostermann M. improved outcome of severe acute pancreatitis in the intensive care unit. Crit Care Res Pract. 2013;2013:897107. [CrossRef] [PubMed]
21. Dellinger EP, Forsmark CE, Layer P, et al. Determinant- based classification of acute pancreatitis severity: an international multidisciplinary consultation. Ann Surg. 2012;256:875-80. [CrossRef] [PubMed]
22. Chan YP, Ning JW, Je F. Establishment of critical period of severe acute pancreatitis associated lung injury. Hepatobiliary Pancreas Dis Int. 2009;8: 535-40. [PubMed]
23. Pastor CM, Matthy MA, Frossard JL. Pancreatitis associated acute lung injury – new insights. Chest. 2003;124: 2341-51. [CrossRef] [PubMed]
24. Lin HY, Lai JI, Lai YC, Lin PC, Chang SC, Tang GJ.Acute renal failure in severe pancreatitis: a population based study. Ups J Med Sci. 2011;116:155-9. [CrossRef] [PubMed]
25. Li XY, Wang XB, Liu XF, Li SG. Prevelence and risk factor of organ failure in patients with severe acute pancreatitis. World J Emerg Med. 2010;1(3):201-4. [PubMed]
26. Frey C, Zhou H, Harvey D. Co-morbidity is a strong predictor of early death andmulti-organ system failure among patients with acute pancreatitis. J Gastrointest Surg. 2007;11:733-42. [CrossRef] [PubMed]
27. Kim BG, Noh MH, Ryu CH, Nam HS, Woo SM, Ryu SH, Jang JS, Lee JH, Choi SR, Park BH. A comparison of the BISAP score and serum procalcitonin for predicting the severity of acute pancreatitis. Korean J Intern Med. 2013;28:322-9. [CrossRef] [PubMed]
28. Chen L, Lu G, Zhou Q, Zhan Q. Evaluation of the BISAP score in predicting severity and prognoses of acute pancreatitis in Chinese patients. Int Surg. 2013;98(1):6-12. [CrossRef] [PubMed]
29. Simchuk EJ, Traverso LW, Nukui Y, Kozarek RA. Computed tomography severity index is a predictor of outcomes for severe pancreatitis. Am J Surg. 2000;179:352-5. [CrossRef] [PubMed]
30. Leung TK, Lee CM, Lin SY, Chen HC, Wang HJ, Shen LK, Chen YY. Balthazar computed tomography severity index is superior to Ranson criteria and APACHE II scoring system in predicting acute pancreatitis outcome. World J Gastroenterol. 2005;11:6049-52. [PubMed]
31. Shinzeki M, Ueda T, Takeyama Y, Yasuda T, Matsumura N, Sawa H, Nakajima T, Matsumoto I, Fujita T, Ajiki T, Fujino Y, Kuroda Y. Predictors of early death in severe acute pancreatitis. J Gastroenterol. 2008;43:152-8. [CrossRef] [PubMed]
32. Lankisch PG, Pflichthofer D, Lehnick D. No strict correlation between necrosis and organ failyre in acute pancreatitis. Pancreas. 2000;20:319-22. [CrossRef] [PubMed]
33. Yang XN, Guo J, Lin ZQ, Huang L, Jin T, Wu W, Wen L, Zhang ZD, Xia Q, Hu WM. The study on causes of death in fulminant pancreatitis at early stage and late stage. Sichuan Da Xue Xue Bao Yi Xue Ban. 2011;42(5):686-90. [PubMed]

Reference as: Maheshwari N, Kumar A, Iqbal ZA, Mandal AK, Vyas A, Wig JD. Organ failure in acute pancreatitis and its impact on outcome in critical care. Southwest J Pulm Crit Care. 2015;10(5):253-64. doi: http://dx.doi.org/10.13175/swjpcc055-15 PDF