Brief Review of Coronavirus for Healthcare Professionals February 10, 2020

Richard A. Robbins, MD¹

Stephen A. Klotz, MD²

¹Phoenix Pulmonary and Critical Care Research and Education Foundation, Gilbert, AZ USA

²Division of Infectious Diseases, Department of Internal Medicine, University of Arizona, Tucson, AZ USA

The epidemic of coronavirus (2019-nCoV) near Wuhan City and the surrounding Hubei Province in China has received extensive news coverage. Some have predicted the virus will cause a worldwide pandemic (1). The CDC has an extensive website discussing over numerous pages whom to suspect, how to diagnose and how to treat 2019-nCoV. 2019-nCoV represents the most recent of the severe coronaviral infections. Severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS) are also caused by coronaviruses that have jumped from animals to humans like 2019-nCoV. It should be remembered that there are only 12 confirmed cases of 2019-nCoV in the US and the mortality rate appears to be only about 3% which is lower than SARS or MERS (2,3). This could be offset by a greater infectiousness of 2019-nCoV resulting in more aggregate infectious, and hence, deaths.

Anyone with a fever who has recently visited the epidemic area in China or been exposed to someone with known 2019-nCoV should be quarantined (2). The only reliable symptom has been fever (98%) (4). Cough (76%), myalgia/fatigue (44%), sputum production (28%), headache (8%), hemoptysis (5%), and diarrhea (3%) were much less common. The clinical course was characterized by the development of dyspnea in 55% of patients and lymphopenia in 66%.

Persons suspected of 2019-nCoV should be quarantined and reported to their local state health departments. The incubation period appears about 2-14 days and is spread by person-to-person transmission based on the previous MERS epidemic (2). There is no need to wear masks in the US where the incidence is low and they are likely ineffective (2).

Diagnosis is made real-time reverse transcription polymerase chain reaction (rRT-PCR) assay. This was only available from the CDC but very recently the CDC has made kits available to state health departments (2).

At present the treatment for 2019-nCoV is supportive in appropriate respiratory isolation to protect healthcare workers. A randomized, controlled trial of Gilead’s antiviral drug remdesivir used to treat Ebola is currently underway in China in hopes that it will be an effective treatment for 2019-nCoV (5).

Please be aware that this information is current as of February 10, 2020. It is likely to change.

References

1. McNeil DG Jr. Wuhan coronavirus looks increasingly like a pandemic, experts say. New York Times. February 2, 2020. Available at: https://www.nytimes.com/2020/02/02/health/coronavirus-pandemic-china.html (accessed 2/10/20).
2. Centers for Disease Control. 2019 Novel Coronavirus (2019-nCoV) in the U.S. February 10, 2020. Available at: https://www.cdc.gov/coronavirus/2019-ncov/cases-in-us.html (accessed 2/10/20).
3. Worldometer. Novel coronavirus (2019-nCoV) mortality rate. Available at: https://www.worldometers.info/coronavirus/coronavirus-death-rate/ (accessed 2/10/20).
4. Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020 Jan 24. pii: S0140-6736(20)30183-5. [Epub ahead of print] [CrossRef] [PubMed]
5. Wetsman N. An experimental antiviral medication might help fight the new coronavirus. The Verge. Feb 4, 2020. Available at: https://www.theverge.com/2020/2/4/21122327/coronavirus-experimental-medication-treatment-wuhan-china-gilead-hiv (accessed 2/10/20).

Cite as: Robbins RA, Klotz SA. Brief review of coronavirus for healthcare professionals February 10, 2020. Southwest J Pulm Crit Care. 2020;20(2):69-70. doi: https://doi.org/10.13175/swjpcc011-20 PDF 

Meta-Analysis of Self-Management Education for Patients with Chronic Obstructive Pulmonary Disease

Jessica Hurley, MD¹

Richard D. Gerkin, MD¹

Bonnie Fahy, RN, MN²

Richard A. Robbins, MD^2* 

Good Samaritan Regional Medical Center¹ and the Phoenix Pulmonary and Critical Care Research and Education Foundation², Phoenix, AZ

Abstract

Background

Chronic obstructive pulmonary disease (COPD) is a common disease frequently associated with high use of health services. Self-management education is a term applied to programs aimed at teaching patients skills that promote the self-efficacy needed to carry out medical regimens specific to control their disease. In COPD, the value of self-management education is not yet clear and a recent trial was terminated early because of excess mortality in the intervention group.

Objectives

The objective of this meta-analysis was to assess the settings, methods and efficacy of COPD self-management education programs on patient outcomes and healthcare utilization.

Selection criteria

Randomized controlled trials of self-management education in patients with COPD were identified. Studies focusing primarily on comprehensive pulmonary rehabilitation (education and exercise) and studies without usual care as a control group were excluded.

Search strategy

We searched PubMed (January 1985 to May 2012) as well as other meta-analysis and reviews.

Data collection and analysis

Two reviewers (JH and RAR) independently assessed study quality and extracted data. Investigators were contacted for additional information.

Main results

The reviewers included 3 group comparisons drawn from 12 trials. The studies showed no significant change in mortality, with one study being an outlier compared to the others.  However, the meta-analysis revealed a reduction in the probability of hospital admission among patients receiving self-management education compared to those receiving usual care.

Conclusions

It is likely that self-management education is associated with a reduction in hospital admissions with no change in mortality. However, because of heterogeneity in interventions, study populations, follow-up time, and outcome measures, data are still insufficient to formulate clear recommendations regarding the preferred curriculum and delivery method of self-management education programs in COPD.

Introduction

Chronic obstructive pulmonary disease (COPD) is currently the third leading cause of death and the only one of the top 5 causes of death that is increasing (1).  The economic and social burden of the disease is immense. The patient usually suffers progressive disability with frequent hospitalizations and emergency room visits. Hospitalizations and emergency room visits account for much of the health care costs from COPD, and therefore, strategies to decrease the these outcomes have received considerable attention (2,3). 

One strategy to improve COPD care has been self-management education, a term applied to any formalized patient education program aimed at increasing knowledge and teaching skills that increase self-efficacy, thus improving collaboration with their healthcare provider to optimally manage patient care. Similar strategies have been successful in other chronic diseases (4-6). However, the effects of self-management programs in COPD, although encouraging, are still unclear (7). Furthermore, a recent trial was terminated prior to enrollment of the planned number of subjects because of excess mortality in the intervention group receiving self-management education (8).

Prompted by the surprising result of an increase in mortality, we reexamined health care outcomes for COPD self-management education by meta-analysis. We found no significant change in mortality but significant reductions in hospitalizations.

Methods

Criteria for considering studies for this review

Types of Studies: Only randomized controlled trials evaluating the effect of self-management education on patients with COPD were used.  Every study included some form of patient education that addressed COPD disease self-management. For inclusion, the study must also include a control group that received usual care and were excluded from the interventional self-management education.  Studies prior to 1985 were not included since medical management for COPD differed from current practice guidelines. 

Types of study participants: Only patients with a clinical diagnosis with COPD were included.  Spirometry was not required to be reported in the study to determine the diagnosis of COPD if the patients admitted had previously been diagnosed with COPD by the referring physician. Patients with a sole diagnosis of asthma or reactive airway disease were excluded from this review.

Types of interventions: In order to qualify as an intervention, the primary goal of the study had to center on improving the patient’s fundamental knowledge and understanding of the disease process and self-management of COPD. The methods of information delivery were highly variable and included written, verbal, visual, and/or audio communication.

Types of outcomes measured: The outcomes identified in studies that were included in this review include mortality, hospital admissions, and emergency room visits.

Search methods

Two separate reviewers (JH, RAR) used systematic searches via the information databases including PubMed.  The terms used to search included “COPD” in addition to one of the following words or phrases: “educat*” or “education” or “patient-educat*” or “patient-education” or “patient educat*” or “patient-education” or “self-manag*” or “self-management” or “self manag*” or “self management” or “disease manag*” or “disease management”.  The searches are current through May of 2012. 

Data collection and analysis

Selection of studies: The two reviewers placed successfully retrieved articles using the above search criteria into 3 categories:

1. Include: RCT evaluating COPD patients and self-management education versus usual care
2. Possibly Include: RCT evaluating COPD patients and disease education but more information needed beyond what is available in the abstract
3. Exclude: not an RCT, not focused on self-management of COPD or did not include usual care comparison or primary outcome focused solely on pulmonary rehabilitation

Data extraction: Information from the accepted studies was collected and included: number of patients in the control and interventional groups, type of intervention used (i.e. disease education, medication instructions, pharmacy action plans), length of study until primary outcome, mortality of each group, respiratory-related hospital admissions, and respiratory-related ED visits.

Data analysis:

Publication bias:  Funnel plots were constructed to examine the pattern of study effects by study size.  Outliers on the plot with respect to a 95% confidence interval were also determined.

Assessment of heterogeneity: The I square statistic was used to examine variability in study results.  If I square was greater than 20%, sensitivity analysis was conducted to determine, if possible, the source of heterogeneity.

Data synthesis:  Continuous outcomes were analyzed using weighted mean difference with 95% confidence intervals.  For dichotomous outcomes, a pooled odds ratio was used.  A fixed effects model was used if I square was less than 20%.  A random effects model, using the technique of DerSimonian and Laird (20), was used if I square was greater than 20 %.

RevMan 5.1. (Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2011) was used for the analysis.

Results

Results of the search: Searches identified 1904 titles and abstracts that were screened to identify 71 potentially relevant articles about self-management education in COPD. Full-text versions of these papers were obtained, and independently assessed by two reviewers (JH and RAR). These were searched for data on mortality, hospitalizations and emergency room (ER) visits.  A total of 12 trials were identified which met the review entry criteria (8-19).

Subjects: A total of 2476 patients were randomized in the 12 studies. The studies were heterogenous with some recruiting patients from outpatient clinics, some from general practice, some from inpatient hospital admissions for COPD exacerbations and some from several sources.

Interventions: All 12 studies described COPD self-management education compared with usual care. The educational delivery mode consisted of group education or individual education. Educational topics varied, as did the discipline of the provider. The follow-up time was variable ranging from 2-12 months.

Comparisons: Twelve studies that compared self-management education with usual care have been included in this review. In one study two intervention groups and one usual care group were used (11). The intervention groups were considered sufficiently similar to be combined.

Outcomes: Reported outcome categories were variable. Studies included in the review identified mortality (10 studies), respiratory-related hospital admissions (9 studies) and emergency room (ER) visits (4 studies).

Missing data: Additional data was requested from the two most recent studies (8,9). A reply was received from one author and is listed in the acknowledgement section.

Mortality: Ten studies reporting mortality were included in the meta-analysis (8-15,18,19). There was no significant difference in mortality between the usual care and intervention groups (OR 0.76; 95% CI (0.44 to 1.30); Figure 1; p=0.31).

Figure 1.  Forest plot of mortality

The level of statistical heterogeneity for this outcome (I square = 54%) may be related to the outlying effect from the report by Fan et al. (8), since its removal led to a lower I square statistic (0%). Also removal of the study resulted in a statistically significant improvement in mortality rate (OR 0.64; 95% CI 0.46 to 0.90)

Respiratory-related Hospital admissions: Nine studies reporting COPD-related hospital admissions were included in the meta-analysis (8-11,13-16,18). There was little heterogeneity present (I square = 0%).  There was a clinically and statistically significant reduction of the probability of at least one hospital admission among patients receiving self-management education compared to those receiving usual care (OR 0.76; 95% CI (0.65 to 0.88); p< 0.001; Figure 2).

 

Figure 2. Forest plot of pulmonary hospitalization

Emergency room visits: Four trials that reported the effect of self-management education on Emergency Room (ER) visits related to COPD were included in the meta-analysis (9,11,12,17). Although the level of heterogeneity was high (I square = 83%), removal of any one study had little effect on this variability.  There was no significant difference between patients receiving self-management education compared to those receiving usual care in the average number of respiratory-related emergency room visits (Mean difference 0.12/pt-yr; 95% CI (-0.21 to 0.46): p=0.47; Figure 3).

Figure 3. Forest plot of pulmonary emergency room visits/pt-yr.

Discussion

This meta-analysis systematically evaluated comparisons of self-management education for patients with COPD compared to usual care. The review was prompted by a recent report of increased mortality in patients receiving COPD education (8). Meta-analysis did not confirm an increase in mortality and determined the recent study had significant heterogeneity compared to the other studies.  We confirm a previous meta-analysis which demonstrated a significant decrease in COPD-related hospitalizations in the intervention groups (7). 

Self-management education has been successfully utilized in a number of chronic diseases (4-6). Education including the use of pre-defined action plans may lead to faster and more frequent treatment of COPD exacerbations, thus resulting in the reduction in hospitalizations. Although we did not review cost-effectiveness, hospitalizations represent the major cost of COPD care (2,3). Therefore, self-management education is likely cost-effective. In support of this concept, a recent cost-effective analysis of one successful self-management education program revealed an average cost savings of $593 per patient (21).

This review has a number of limitations. First, there was variation in the intervention content and delivery. Some studies included action plans in the self-management curriculum and others incorporated additional components of pulmonary rehabilitation including exercise. The type and intensity of education delivery varied from one-on-one instruction, group interaction and the distribution of written material.

Second, the COPD-population was defined in varying detail, with studies using very diverse inclusion criteria. As a result, heterogeneity in disease severity was present. This may explain some of the differing results, including the increase in mortality observed in the recently published study (8).

Third, the studies assessed a broad spectrum of outcome measures and length of follow-up. Often meta-analyses could not be performed because of different outcome measures utilized or different methodology used to calculate the same outcome (e.g. ER visits). This lack of data consistency hampered statistical combination and therefore may have biased the estimates in the review.  Since self-management programs are intended to achieve behavioral changes, follow-up should ideally be long term and this was not the case in all studies.

The final limitation was that knowledge of one’s disease does not necessarily lead to behavioral change. It is unclear at this point if the educational programs lead to an increase in healthy behaviors.

The results of the study by Fan et al. (8) showing an increase in mortality is not confirmed by this meta-analysis. Fan’s manuscript describes the BREATH trial which was a randomized, controlled, multi-center trial performed at 20 VA medical centers comparing an educational comprehensive care management program to guideline-based usual care for patients with chronic obstructive pulmonary disease. The intervention included COPD education during 4 individual and 1 group sessions, an action plan for identification and treatment of exacerbations, and scheduled proactive telephone calls for case management. It is unclear why this education and self-management which is not very dissimilar from other studies would increase mortality. Although the patients were recruited after they were hospitalized, and therefore, likely had more advanced COPD than in some other studies, this alone should not explain excess mortality in the intervention group. An accompanying editorial by Pocock in the same issue of the Annals of Internal Medicine identified no apparent reason for the increase in mortality and points out that education seems an unlikely cause (22). We also have been unable to identify an explanation for the increase and agree with Pocock that the reason seems most likely secondary to statistical chance. The present meta-analysis is consistent with this concept.

For future research of the efficacy of self-management education of COPD patients in improving patient outcomes and decreasing health care utilization, it is important to create more homogeneity in the design of the studies (educational curriculum, demographics, outcome measures and follow-up period). The effectiveness of the individual components of self-management education programs (i.e., action plans, exercise programs) should also be evaluated.

From this meta-analysis, we have shown that self-management education is associated with a reduction in hospital admissions, with no indication for detrimental effects in other outcome parameters. This would seem sufficient to justify a recommendation of self-management education in COPD. However, due to diversity in interventions, study populations, follow-up time, and outcome measures, data are still insufficient to formulate clear recommendations regarding the form and content of self-management education programs in COPD.

Acknowledgements

We are grateful to Kathryn Rice for her assistance in obtaining additional data from her study (9).

References

1. Akinbami LJ, Liu X. Chronic obstructive pulmonary disease among adults aged 18 and over in the United States, 1998-2009. NCHS Data Brief 2011;63:1-8.
2. Toy EL, Gallagher KF, Stanley EL, Swensen AR, Duh MS. The economic impact of exacerbations of chronic obstructive pulmonary disease and exacerbation definition: a review. COPD 2010;7:214-28.
3. Hilleman DE, Dewan N, Malesker M, Friedman M. Pharmacoeconomic evaluation of COPD. Chest 2000;118:1278-85.
4. Ofman JJ, Badamgarav E, Henning JM, Knight K, Gano AD, Jr., Levan RK, et al. Does disease management improve clinical and economic outcomes in patients with chronic diseases? A systematic review. Am J Med 2004;117:182-92.
5. Gwadry-Sridhar FH, Flintoft V, Lee DS, Lee H, Guyatt GH. A systematic review and meta-analysis of studies comparing readmission rates and mortality rates in patients with heart failure. Arch Intern Med 2004;164:2315-20.
6. Jovicic A, Holroyd-Leduc JM, Straus SE. Effects of self-management intervention on health outcomes of patients with heart failure: a systematic review of randomized controlled trials. BMC Cardiovasc Disord 2006;6:43.
7. Effing T, Monninkhof EM, van der Valk PD, van der Palen J, van Herwaarden CL, Partidge MR, Walters EH, Zielhuis GA. Self-management education for patients with chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2007;17:CD002990.
8. Fan VS, Gaziano JM, Lew R, et al. A comprehensive care management program to prevent chronic obstructive pulmonary disease hospitalizations: a randomized, controlled trial. Ann Intern Med 2012;156:673-83.
9. Rice KL, Dewan N, Bloomfield HE, Grill J, Schult TM, Nelson DB, Kumari S, Thomas M, Geist LJ, Beaner C, Caldwell M, Niewoehner DE. Disease management program for chronic obstructive pulmonary disease: a randomized controlled trial. Am J Respir Crit Care Med. 2010;182:890-6.
10. Boxall A, Barclay L, Sayers A, Caplan GA. Managing chronic obstructive pulmonary disease in the community. A randomized controlled trial of home-based pulmonary rehabilitation for elderly housebound patients. J Cardiopulm Rehabil 2005;25:378–85.
11. Coultas D, Frederick J, Barnett B, Singh G, Wludyka P. A randomized trial of two types of nurse-assisted home care for patients with COPD. Chest 2005;128:2017–24.
12. Martin IR, McNamara D, Sutherland FR, Tilyard MW, Taylor DR. Care plans for acutely deteriorating COPD: a randomized controlled trial. Chronic Respiratory Disease 2004;1:191–5.
13. Rea H, McAuley S, Stewart A, Lamont C, Roseman P, Didsbury P. A chronic disease management programme can reduce days in hospital for patients with chronic obstructive pulmonary disease. Intern Med J 2004;34:608–14.
14. Bourbeau J, Julien M, Maltais F, et al. Reduction of hospital utilization in patients with chronic obstructive pulmonary disease: a disease specific self-management intervention. Arch Intern Med 2003;163:585–91.
15. Monninkhof E, van der Valk P, van der Palen J, van Herwaarden C, Zielhuis G. Effects of a comprehensive self-management programme in patients with chronic obstructive pulmonary disease. Eur Respir J 2003;22:815–20.
16. Gallefoss F, Bakke PS, Rsgaard PK. Quality of life assessment after patient education in a randomized controlled study on asthma and chronic obstructive pulmonary disease. Am J Respir Critical Care Med 1999;159:812–7.
17. Gourley GA, Portner TS, Gourley DR, et al. Humanistic outcomes in the hypertension and COPD arms of a multicenter outcomes study. J Am Pharm Assoc 1998;38:586–597.
18. Littlejohns P, Baveystock CM, Parnell H, Jones P. Randomised controlled trial of the effectiveness of a respiratory health worker in reducing impairment, disability, and handicap due to chronic airflow limitation.  Thorax 1991;46:559–64.
19. Cockcroft A, Bagnall P, Heslop A, et al.Controlled trial of respiratory health worker visiting patients with chronic respiratory disability. BMJ (Clin Res Ed) 1987;294:225–8.
20. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-88.
21. Dewan NA, Rice KL, Caldwell M, Hilleman DE. Economic evaluation of a disease management program for chronic obstructive pulmonary disease. COPD 2011;8:153-9.
22. Pocock SJ. Ethical dilemmas and malfunctions in clinical trials research. Ann Intern Med 2012;156:746-747.

Reference as: Hurley J, Gerkin RD, Fahy B, Robbins RA. Meta-analysis of self-management education for patients with chronic obstructive pulmonary disease. Southwest J Pulm Crit Care 2012;4:194-202. (Click here for a PDF version of the manuscript)

For the accompanying editorial "A Little Knowledge is a Dangerous Thing" click here.

Relationship between the Veterans Healthcare Administration Hospital Performance Measures and Outcomes

Richard A. Robbins, M.D.¹

Richard Gerkin, M.D.²

Clement U. Singarajah, M.D.¹

¹Phoenix Pulmonary and Critical Care Medicine Research and Education Foundation and ²Banner Good Samaritan Medical Center, Phoenix, AZ

Reference as: Robbins RA, Gerkin R, Singarajah CU. Relationship between the Veterans Healthcare Administration hospital performance measures and outcomes. Southwest J Pulm Crit Care 2011;3:92-133. (Click here for PDF version of manuscript)

Abstract

Health care organizations have been using performance measures to compare hospitals. However, it is unclear if compliance with these performance measures results in better healthcare outcomes. We examined compliance with acute myocardial infarction, congestive heart failure, pneumonia and surgical process of care measures with traditional outcome measures including mortality rates, morbidity rates, length of stay and readmission rates using the Veterans Healthcare Administration Quality and Safety report. Disappointingly, increased compliance with the performance measures was not correlated with better outcomes with the single exception of improved mortality with higher rates of compliance with echocardiography. We also evaluated the hospital level of care and found that higher levels of complexity of care correlated with the acute myocardial infarction performance measure, but not with the congestive heart failure, pneumonia, or surgical process of care performance measures.  However, level of complexity of care strongly correlated with all cause mortality (p<0.001), surgical mortality (p=0.037) and surgical morbidity (p=0.01). These data demonstrate that compliance with the performance measures are not correlated with improved healthcare outcomes, and suggest that if measures are used to compare hospitals, different measures need to be developed.

Introduction

The Joint Commission recently released “Improving America’s Hospitals: The Joint Commission’s Annual Report on Quality and Safety 2011 (1).  In this report the results of hospital compliance with the Joint Commission’s performance measures are listed. The Joint Commission announced not only is compliance improving but identified 405 hospitals as their “Top Performers on Key Quality Measures Program”. In a letter at the beginning of the report Mark Chassin, President of the Joint Commission, said “This program is designed to be an incentive for better performance on accountability measures and to support organizations in their quest to do better”.

However, there have been several criticisms of the report. First, many hospitals which were recognized as top hospitals by US News & World Report, HealthGrades Top 50 Hospitals, or Thomson Reuters Top Cardiovascular Hospitals were not included (2). Small community hospitals were overrepresented and large academic medical centers were underrepresented in the report. Chassin commented that this should be "a wake-up call to larger hospitals to put more resources into these programs…”. This is surprising since teaching hospitals, which are usually large, urban hospitals, have previously been reported to have lower risk-adjusted mortality rates and lengths of stay (3). Second, it has been pointed out that many of the performance measures are not or only weakly associated with traditional outcomes such as mortality (4-7). Therefore, we compared the compliance with the Joint Commission performance measures compared to mortality rates, morbidity rates, length of stay and readmissions using the Nation’s largest healthcare system, the Department of Veterans Affairs. The results demonstrate that compliance with performance measures are not correlated with improved outcomes.

Methods

The study was approved by the Western IRB.

Process Performance Measures. We evaluated hospital performance based on publicly available data from the 2010 VHA Facility Quality and Safety Report (9). These measures evaluate quality of care for acute myocardial infarction, congestive heart failure, pneumonia and surgical care improvement program (SCIP) during fiscal year 2009. For each of the measures, a hospital’s performance is calculated as the proportion of patients who received the indicated care out of all the patients who were eligible for the indicated care. The quality indicators are based on, and in most cases identical to those used for the Joint Commission’s Hospital Compare (acute myocardial infarction-Appendix 1; congestive heart failure-Appendix 2; pneumonia-Appendix 3, surgical quality-Appendix 4). Data were also available for each component of the congestive heart failure quality measure (see Appendix 2) which was evaluated independently.

Disease specific mortality. Hospital-specific, risk-standardized rates of mortality within 30 days of discharge are reported for patients hospitalized with a principal diagnosis of heart attack, heart failure, and pneumonia. For each condition, the risk-standardized (also known as "adjusted" or "risk-adjusted") hospital mortality rate are calculated using mathematical models that use administrative data to adjust for differences in patient characteristics that affect expected mortality rates (10).

Surgical morbidity and mortality. VA’s Surgical Quality Improvement Program (VASQIP) monitors major surgical procedures performed at VHA facilities and tracks risk adjusted surgical complications (morbidity) and mortality rates. Patient data are collected at each facility by a specially trained nurse and entered into the VA’s electronic health record: detailed preoperative patient characteristics including chart-abstracted medical conditions, functional status, recent laboratory tests, information about the surgical procedure performed, and 30-day outcomes data.

The VASQIP program analyzes these patient data using mathematical models to predict an individual patient’s expected outcome based on the patient’s preoperative characteristics and the type and nature of the surgical procedure. Overall patient outcomes for major surgical procedures are expressed by comparing observed rates of mortality and morbidity to the expected rates for those patients undergoing the procedure as observed-to-expected (O/E) ratios. For example, if, based on patient characteristics, a facility expected 5 deaths following major surgery, but only 4 patients died, the O/E ratio would be reported as 0.8.

Medical Surgical Length of Stay (LOS). These data are the VA hospital average length of stay for patients who were discharged from acute medicine or surgery bed sections. It does not include patients discharged from observation beds or discharged from other areas of the hospital such as mental health.

Readmission rates. A readmission was defined as a patient who has had a recent hospital stay and needs to re-enter the hospital again within 30 days. These rates are not adjusted for patient characteristics that affected expected admission rates, so comparisons among hospitals should be interpreted with caution.

CHF readmissions were reported separately. CHF readmission is defined by patients who had an initial hospitalization for CHF and were readmitted at least once to acute care in the hospital within 30 days following discharge for CHF.

Hospital level of care. For descriptive purposes, hospitals were grouped into levels of care. These are classified into 4 levels: highly complex (level 1); complex (level 2); moderate (level 3), and basic (level 4). In general, level 1 facilities and some level 2 facilities represent large urban, academic teaching medical centers.

Correlation with Outcomes. Pearson’s correlation coefficient was used to assess the correlation of compliance with the performance measures and outcomes. Significance was defined as p<0.05. For comparisons among hospital levels, ANOVA or Kruskall-Wallis testing was done, as appropriate.

Results

Disease specific and all cause mortality rates compared to performance measures. Hospital-specific, risk-standardized rates of mortality within 30 days of discharge for patients hospitalized with a principal diagnosis of heart attack, heart failure, and pneumonia were compared to performance measure compliance. There was no correlation (Table 1, p>0.05 all conditions) but with an increased incidence of pneumonia actually weakly correlating with higher compliance with the pneumonia performance measures (Table 1, p=0.0411). Furthermore, there was no correlation between all cause mortality and the average of the three compliance measures (Table 1, p>0.05). Because each table is large, only the correlation coefficients are presented in the text. The table data on which the correlations are based are given at the end of the manuscript. (N=the number of hospitals. NA=not available).

Table 1. Disease Specific Mortality Correlated with Performance Measure Compliance

Correlation Coefficients	r value	N	p value
Acute Myocardial Infarction Mortality and AMI Performance Measure	0.0266	103	0.7897
Congestive Heart Failure Mortality and CHF Performance Measure	0.0992	123	0.2752
Pneumonia Mortality and Pneumonia Performance Measure	0.1844	123	0.0411
All Cause Mortality vs. Average of Performance Measures	0.1118	122	0.2202

Each component of the congestive heart failure performance measure was evaluated individually. Performance of echocardiography correlated with improved mortality (Table 2, p=0.0496) but there was no correlation with use of a angiotensin converting enzyme inhibitor (ACEI) or angiotensin receptor blocker (ARB) at discharge, discharge instructions, nor smoking cessation advice (Table 2, p>0.05 all comparisons).

Table 2. Heart Failure Mortality Correlated with Compliance to Individual Heart Failure Performance Measures

Correlation Coefficients	r value	N	p
ACEI or ARB	-0.1007	112	0.2908
Smoking Cessation	0.0651	112	0.4953
Discharge Instructions	0.1411	111	0.1396
Echocardiography	-0.1860	112	0.0496

Surgical mortality and morbidity rates compared to surgical performance measures. There was no correlation between compliance with the surgical care improvement program (SCIP) and surgical mortality or morbidity (Table 3, p>0.05 both comparisons).

Table 3. Surgical Care Improvement Program (SCIP) Compliance Correlated with Observed/Expected (O/E) Morbidity/Mortality

Correlation Coefficients	r value	N	p value
O/E Mortality	0.0943	99	0.3530
O/E Morbidity	0.0031	99	0.9757

Length of Stay. None of the performance measures correlated with medical-surgical length of stay (Table 4, p>0.05 all comparisons).

Table 4. Length of Stay (LOS) Correlated with Performance Measure Compliance

Correlation Coefficients	r value	N	p value
LOS compared to AMI	0.1047	103	0.2926
LOS compared to CHF	-0.0178	123	0.8451
LOS compared to Pneumonia	-0.1679	123	0.0634
LOS compared to SCIP	-0.0404	106	0.6809
LOS compared to Average	0.0028	123	0.9755

Readmission rates. There was no correlation between all cause readmission rates and the acute myocardial infarction, congestive heart failure, pneumonia or surgical performance measures (Table 5, p>0.05 all comparisons). There was no correlation between heart failure readmission rate and the heart failure performance measure (data not shown, r=0.1525, p=0.0921).

Table 5. Readmission Rate Correlated with Performance Measure Compliance

Correlation Coefficients
	r value	N	p
AMI	0.1688	103	0.0883
CHF	0.1505	123	0.0966
Pneumonia	0.0581	123	0.5233
Average	0.1281	122	0.1597

Hospital level of care. Acute myocardial infarction performance measures inversely correlated with the hospital level of care, i.e., the higher the hospital complexity level, the better the compliance (Table 6, p=0.004). However, there was no correlation between congestive heart failure, pneumonia, surgical care improvement program or the average of the measures and the hospital level of care (Table 6).

Table 6. Hospital Level Correlated with Performance Measure Compliance

ANOVA	N	p
Acute Myocardial Infarction (AMI)	103	0.004
Congestive Heart Failure (CHF)	120	0.782
Community Acquired Pneumonia	120	0.296
Surgical Care Improvement Program (SCIP)	106	0.801
Average of Process of Care Measures	120	0.285

There was no correlation between the level of hospital care and the acute myocardial infarction, congestive heart failure, nor pneumonia mortality (Table 7, p>0.05 all comparisons). However, there was a strong correlation between all cause morality (p<0.001) and a correlation between surgical Observed/Expected mortality (Table 7, p=0.037) and surgical Observed/Expected morbidity (p=0.010).

Table 7. Hospital Level Correlated with Mortality and Surgical Morbidity

ANOVA	N	p
Acute Myocardial Infarction (AMI) Mortality	103	0.835
Congestive Heart Failure (CHF) Mortality	120	0.493
Pneumonia Mortality	120	0.547
All Cause Mortality	106	<0.001
Surgical O/E Mortality	99	0.037
Surgical O/E Morbidity	99	0.010

Discussion

These data from the Nation’s largest healthcare system demonstrate that increasing compliance of the performance measures prescribed by the Joint Commission does not affect disease specific mortality, all cause mortality, surgical mortality, surgical morbidity, length of stay or readmissions with the single exception of improved mortality correlating with increased compliance with performance of echocardiography. In contrast to the Joint Commission’s list of top hospitals which found smaller and rural hospitals to be overrepresented, we found that only the acute myocardial infarction performance measure correlated with a higher level of hospital care which represents mostly large, urban hospitals. We did find that all cause mortality and surgical morbidity highly correlated with the level of care. This would appear to differ from the Joint Commission’s list of top hospitals which tended to be small and rural, since VA hospitals with higher levels of care largely represent large urban, academic teaching medical centers.

There are multiple possible reasons for the lack of correlation between the performance measures and outcomes. Many of the outcomes are evidence based but several are not. For example, there are no randomized, multi-center studies evaluating the efficacy of discharge instructions, smoking cessation advice and pneumococcal vaccination. Studies with discharge instructions are retrospective, observational studies and have largely not shown improved outcomes (11,12). Several meta-analyses have failed to demonstrate the efficacy of pneumococcal vaccine in adults (13-15). Advice to quit smoking without follow up support or pharmacologic intervention has not been shown to lower smoking cessation rates (16). Mandating ineffective interventions such as these would not be expected to have a positive effect on outcomes. However, this is where most of the improvement in performance measure outcome has occurred (2).

Most of the interventions are grouped or bundled. Lack of compliance with any one of the bundle is taken as noncompliance with the whole. However, if the only difference between hospitals is noncompliance with an ineffective performance measure, there would not be any expected improvement in outcomes.

Many of the strongly evidence-based outcomes have very high compliance, usually exceeding 95% (9). It is possible that small improvements of 1 or 2% in effective performance measures might have too small an impact on outcomes to be detected even in large databases such as the Veterans Administration which examined 485,774 acute medical/surgical discharges in 2009.

The performance measures appear to avoid highly technical or costly interventions and often avoid interventions which have been shown positively affect outcomes. For example, beta blockers and spironolactone have been shown to be effective in heart failure but are not included in the congestive heart failure performance measures (17,18). Furthermore, carvedilol has been shown to be superior to metoprolol in improving survival (19). Why the performance measures include use of an angiotensin converting enzyme inhibitor or angiotensin receptor blocker but not carvedilol and spironolactone is unclear.

Some of the performance measures may have caused inadvertent harm. For example, administration of antibiotics within 4 hours to patients with pneumonia was a previous performance measure. However, studies showed that this performance measurement led to administration of antibiotics in many patients who proved not to have pneumonia or another infectious disease, and a systematic review concluded that “evidence from observational studies fails to confirm decreased mortality with early administration of antibiotics in stable patients with [community acquired pneumonia]”  (20-22). The time has since been changed to 6 hours, but it is unclear if that it is any better than the initial 4 hour timing used (7).

We did not confirm the Joint Commission’s findings that the top hospitals are overrepresented by small, rural hospitals. We found no correlation between hospital level of complexity of care and performance measure compliance with the exception of acute myocardial infarction which was higher in hospitals with higher complexities of care. Although we found no correlation of the performance measures with any outcome measures, we did find a strong correlation between the hospital level of complexity of care and overall survival and surgical morbidity with the hospitals having the higher level of complexity having improved survival and decreased surgical morbidity. This would seem consistent with concept that volume of care correlates with outcomes.

It seems surprising that initiation of performance measures seem to go through such little scrutiny. In a 2005 editorial Angus and Abraham (23) addressed the issue of when there is sufficient evidence for a concept to be widely applied as a guideline or performance measure. Comparing guidelines to evaluation of novel pharmacologic therapies, they point out that promising phase II studies are insufficient for regulatory approval. Instead, one, and usually two, large multicenter phase III trials are necessary to confirm reliability. The same principle is echoed in evidence-based medicine, where grade A recommendations are based on two or more large, positive, randomized, and multicenter trials. This seems a reasonable suggestion. Perhaps what is needed is an independent Federal or private agency to review and approve performance measures, and as Angus and Abraham suggest, require at least two randomized, multicenter trials before implementation

The data presented in this manuscript do not support the usefulness of increasing compliance with the Veterans Administration’s (or the Joint Commission’s) performance measures in improving outcomes such as mortality, morbidity, length of stay or readmission rates. Until compliance with the performance measures results in improved outcomes, investment to improve these performance measures seems to be a poor utilization of resources. It suggests that oversight of regulatory agencies is needed in developing and implementing performance measures. If performance measures are to be used, new, clinically meaningful measures that correlate with outcomes need to be developed. 

References

1. Available at: http://www.jointcommission.org/accreditation/hospitals.aspx (accessed 9-25-11).
2. Available at: http://www.forbes.com/sites/davidwhelan/2011/09/20/is-the-joint-commission-list-of-top-hospitals-worth-heeding/ (accessed 9-25-11).
3. Rosenthal GE, Harper DL, Quinn LM. Severity-adjusted mortality and length of stay in teaching and nonteaching hospitals. JAMA 1997;278:485-90.
4. Werner RM, Bradlow ET. Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 2006;296:2694-702.
5. Peterson ED, Roe MT, Mulgund J, DeLong ER, Lytle BL, Brindis RG, Smith SC Jr, Pollack CV Jr, Newby LK, Harrington RA, Gibler WB, Ohman EM. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912-20.
6. Fonarow GC, Yancy CW, Heywood JT; ADHERE Scientific Advisory Committee, Study Group, and Investigators. Adherence to heart failure quality-of-care indicators in US hosptials: analysis of the ADHERE Registry. Arch Int Med 2005;165: 1469-77.
7. Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med 2008;149:29-32.
8. Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM.  Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303:2479-85.
9. Available at: http://www.va.gov/health/docs/HospitalReportCard2010.pdf (accessed 9-28-11).
10. Ross JS, Maynard C, Krumholz HM, Sun H, Rumsfeld JS, Normand SL, Wang Y, Fihn SD. Use of administrative claims models to assess 30 day mortality among Veterans Health Administration hospitals. Medical Care 2010; 48: 652-658.
11. VanSuch M, Naessens JM, Stroebel RJ, Huddleston JM, Williams AR. Effect of discharge instructions on readmission of hospitalised patients with heart failure: do all of the Joint Commission on Accreditation of Healthcare Organizations heart failure core measures reflect better care? Qual Saf Health Care 2006;15:414-7.
12. Fonarow GC, Abraham WT, Albert NM, Stough WG, Gheorghiade M, Greenberg BH, O'Connor CM, Pieper K, Sun JL, Yancy C, Young JB; OPTIMIZE-HF Investigators and Hospitals. Association between performance measures and clinical outcomes for patients hospitalized with heart failure. JAMA 2007;297:61-70.
13. Fine MJ, Smith MA, Carson CA, Meffe F, Sankey SS, Weissfeld LA, Detsky AS, Kapoor WN. Efficacy of pneumococcal vaccination in adults. A meta-analysis of randomized controlled trials. Arch Int Med 1994;154:2666-77.
14. Dear K, Holden J, Andrews R, Tatham D. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Sys Rev 2003:CD000422.
15. Huss A, Scott P, Stuck AE, Trotter C, Egger M. Efficacy of pneumococcal vaccination in adults: a meta-analysis. CMAJ 2009;180:48-58.
16. Rigotti NA, Munafo MR, Stead LF. Smoking cessation interventions for hospitalized smokers: A systematic review. Arch Intern Med 2008;168:1950-1960.
17. Gottlieb SS, McCarter RJ, Vogel RA. Effect of beta-blockade on mortality among high-risk and low-risk patients after myocardial infarction. N Engl J Med 1998;339:489-97.
18. Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, Perez A, Palensky J, Wittes J for the Randomized Aldactone Evaluation Study Investigators. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med 1999;341:709-17.
19. Poole-Wilson PA, Swedberg K, Cleland JG, Di Lenarda A, Hanrath P, Komajda M, Lubsen J, Lutiger B, Metra M, Remme WJ, Torp-Pedersen C, Scherhag A, Skene A. Carvedilol Or Metoprolol European Trial Investigators. Comparison of carvedilol and metoprolol on clinical outcomes in patients with chronic heart failure in the Carvedilol or Metoprolol European Trial (COMET): randomised controlled trial. Lancet. 2003;362:7-13.
20. Kanwar M, Brar N, Khatib R, Fakih MG. Misdiagnosis of community acquired pneumonia and inappropriate utilization of antibiotics: side effects of the 4-h antibiotic administration rule. Chest 2007;131:1865-9.
21. Welker JA, Huston M, McCue JD. Antibiotic timing and errors in diagnosing pneumonia. Arch Intern Med 2008;168:351-6.
22. Yu KT, Wyer PC. Evidence-based emergency medicine/critically appraised topic. Evidence behind the 4-hour rule for initiation of antibiotic therapy in community-acquired pneumonia. Ann Emerg Med 2008;51:651-62.
23. Angus DC, Abraham E. Intensive insulin therapy in critical illness: when is the evidence enough? Am J Resp Crit Care 2005;172:1358-9

Click here for Excel version of Table 1

Click here for Excel version of Table 2

Click here for Excel version of Table 3

Click here for Excel version of Table 4

Click here for Excel version of Table 5

Click here for Excel version of Table 6

Click here for Excel version of Table 7

Click here for Appendix 1

Click here for Appendix 2

Click here for Appendix 3

Click here for Appendix 4