Improving Quality in Healthcare

Figure 1. Dr. Katz is a little jaded about quality metrics (1).

Everyone is in favor of quality healthcare and improving it. However, to date, initially highly touted quality measures prove to be meaningless metrics in about 5-10 years. That is, when the measures are scientifically studied, they are found to be of little worth. The cycle is then repeated, i.e., new and highly touted measures are again selected and found to be useless in 5-10 years. The latest in this cycle may be the Centers for Medicare and Medicaid’s (CMS) Merit-based Incentive Payment System (MIPS). The theory underlying MIPS has been that paying for quality rather than quantity will incentivize healthcare providers to improve quality. As part of the deal creating the Affordable Care Act (Obamacare) MIPS was established as a pay for performance system which promised to improve healthcare while reducing costs. However, healthcare costs have continued to rise (2). Data on improvement in quality has been lacking.

Now, Bond et al. (3) have reported a study suggesting that MIPS incentivization of quality improvement in healthcare quality has questionable benefits. Among US primary care physicians in 2019, MIPS scores were inconsistently associated with performance on process and outcome measures. Bond’s study included 3.4 million patients attributed to 80,246 primary care physicians. Physicians were divided into thirds based on their MIPS score. Compared with physicians with high MIPS scores, physicians with the lowest MIPS scores had significantly worse mean performance on 3 of 5 process measures: diabetic eye examinations, diabetic HbA1c screening and mammography screening, but significantly better mean performance on rates of influenza vaccination and tobacco screening. MIPS scores were inconsistently associated with risk-adjusted patient outcomes: compared with physicians with the highest MIPS scores, physicians with the lowest MIPS scores had significantly better mean performance on emergency department visits per 1000 patients but worse performance on all-cause hospitalizations, and did not have significantly different performance on 4 ambulatory care-sensitive admission outcomes. Nineteen percent of physicians with the lowest MIPS scores had composite outcomes performance in the top quintile, while 21% of physicians with the highest MIPS scores had outcomes in the bottom quintile. These findings suggest that the MIPS program may be ineffective at measuring and incentivizing quality improvement among US physicians.

It is unclear why improvement  in intermediate surrogate markers is used rather than improvement in outcomes. Bond’s study measured MIPS scores against ER visits and hospitalizations. Patients, providers, insurers, bureaucrats, politicians, taxpayers- in other words, nearly everyone- would agree that reductions in ER visits and hospitalizations is desirable if it can be accomplished without patient harm. Similarly, reduction in unexpected deaths and improvement in patients’ feeling of well being are goals that all can support. However, the goals of healthcare are different depending on which population is asked. Patients might support their well-being, insurance cost, and provider access as being most important, whereas payors might support costs as most important. Providers might support efficiency of care and reimbursement as important. So ultimately what surrogate markers like MIPS do is choose one point of view which often does not affect outcomes (4).

There are many ways to achieve a goal depending on expertise, resources and patient characteristics. Flexibility in care allows the person most likely to understand the efficiencies of their particular system- the providers- to use their local knowledge to benefit the patients. Outside influences emphasizing surrogate markers, cost, or politics have historically failed. Unless one is willing to accept healthcare shown not to benefit patients as acceptable, MIPS should be eliminated. Replacing MIPS with an equally flawed system set of surrogate markers will likely not help.

It seems that outcome measures offer several advantages over process measures. Outcome measures include unexpected mortality, hospital readmissions, safety of care, effectiveness of care, timeliness of care, efficiency of care, and patient well-being (5). These are all thought to be important by patients, insurers, providers and even politicians. In my view, the process leading to these ultimate outcome goals is less important and the process producing the same or similar results will likely vary between providers and hospitals.

CMS should refocus their quality efforts on outcomes rather than processes which have failed as quality indicators. Physicians must decide whether they wish to continue participation in systems such as MIPS and the accompanying increase in paperwork. Unless something changes the trends of increasing paperwork over meaningless metrics will continue.

Richard A. Robbins MD

Editor, SWJPCCS

References

1. Lehmann C. Comics for Docs: Medical Cartoons Poke Fun at Today's Practices. Medscape. July 15, 2022. Available at: https://www.medscape.com/slideshow/medical-cartoons-6015473#2 (accessed (1/12/23).
2. Kurani N, Ortaliza J, Wager E, Fox L, Amin K. How Has U.S. Spending on Healthcare Changed Over Time? Peterson-KFF Health System Trasecker. February 25, 2022. Available at: https://www.healthsystemtracker.org/chart-collection/u-s-spending-healthcare-changed- time/#Total%20national%20health%20expenditures,%20US%20$%20Billions,%201970-2020 (Accessed 1/4/23).
3. Bond AM, Schpero WL, Casalino LP, Zhang M, Khullar D. Association Between Individual Primary Care Physician Merit-based Incentive Payment System Score and Measures of Process and Patient Outcomes. JAMA. 2022 Dec 6;328(21):2136-2146. [CrossRef] [PubMed]
4. Robbins RA, Thomas AR, Raschke RA. Guidelines, recommendations and improvement in healthcare. Southwest J Pulm Crit Care. 2011;2:34-37.
5. Tinker A. The Top Seven Healthcare Outcome Measures and Three Measurement Essentials. Health Catalyst. June 29, 2022. Available at: https://www.healthcatalyst.com/insights/top-7-healthcare-outcome-measures (accessed 1/5/23).

Cite as: Robbins RA. Improving Quality in Healthcare. Southwest J Pulm Crit Care Sleep. 2023;26(1):8-10. doi: https://doi.org/10.13175/swjpccs002-23 PDF

Medicare for All-Good Idea or Political Death?

Several Democratic presidential candidates have pushed the idea of “Medicare for All” and a “Medicare for All” bill has been introduced into the US house with over 100 sponsors. A recent Medpage Today editorial by Milton Packer asks whether this will benefit patients or physicians (1). Below are our views on “Medicare for All” with the caveat that we do not speak for the American Thoracic Society nor any of its chapters.

It has been repeatedly pointed out that medical care in the US costs too much. US health care spending grew 3.9 percent in 2017, reaching $3.5 trillion or $10,739 per person, and 17.9% of the gross domestic product (GDP) (2). This is more than any industrialized country. Furthermore, our expenditures continue to rise faster than most other comparable countries such as Japan, Germany, England, Australia and Canada (2).

Despite the high costs, the US does not provide access to healthcare for all of its citizens. In 2017, 8.8 percent of people, or 28.5 million, did not have health insurance at any point during the year (3). In contrast, other comparable industrialized countries provide at least some care for everyone.

Furthermore, our outcomes are worse. Infant mortality is higher than any similar country (4). US life expectancy is shorter at 78.6 years compared to just about any comparable industrialized company with Japan leading the way at 84.1 years. All the Western European countries (such as Germany, France, England, etc.), as well as Australia and Canada have a longer life expectancy than the US (range 81.8-83.7 years).

Our high infant mortality and lagging life expectancy was not always so. In 1980, the US had similar infant mortality and life expectancy when compared to other industrialized countries. Why did we lose ground over the last 40 years? Beginning in about 1980, there have been increasing business pressures on our healthcare system. In his editorial, Packer called our system "financialized" to an extreme (1). Hospitals, pharmaceutical and device companies, insurance companies, pharmacies and sadly,  even some physicians often price their products and services not according to what is fair or good for patients but to maximize profit. By incentivizing procedures that often do not benefit patients but benefit the businessmen’s’ pockets, these practices likely account for the high costs and for our worsening outcomes.

Packer points out that in the US, intermediaries (insurers and pharmacy benefit managers) exert considerable control of payment while unnecessarily adding to the administrative costs of healthcare. Congress has been pressured to forbid Medicare from negotiating prices with pharmaceutical companies benefitting only the drug manufacturers and those that benefit from the high drug prices. Consequently, administrative costs are four times higher and pharmaceuticals three times greater in the U.S. than in other countries.

If “Medicare for All” could reduce healthcare costs and improve outcomes, it might seem like a good idea. It has the potential for reducing administrative costs and assuming the power to negotiate drug prices was restored, pharmaceutical costs. However, it will be opposed by those who financially benefit from the present system including administrators, hospitals, pharmaceutical companies, pharmacy benefit managers, insurance companies, etc. Furthermore, there is a libertarian segment of the population that opposes any Government interference in healthcare, even those that would strengthen the free market principles that so many libertarians tout. There are already TV adds opposing “Medicare for All.” It seems likely that any “Medicare for All” or any similar plan will meet with considerable political opposition. 

One solution might be to have both Government and non-Government plans. Assuming transparency in both services covered and costs, it leaves the choice in healthcare plans where it belongs-with those paying for the care. It also makes it much harder for those with financial or political interests to convincingly argue against a Government plan (although we are sure they will try). It will force insurance companies to reduce their prices and/or offer more coverage, which is not a bad thing for patients and ultimately, the healthcare system as a whole. However, it does impose a risk, i.e., that profit-driven insurance companies and those who benefit from the current infrastructure will be  replaced by bureaucrats who are primarily concerned with administrative procedure rather than patient care. Present day examples include the VA, Medicare and Medicaid systems. Close public and medical oversight of such a system would be needed.

Ideally, a healthcare system should ensure that citizens can access at least a basic level of health services without incurring financial hardship and with the goal of improving health outcomes. Such a system, would provide a middle path between the extremes of paying for nothing and paying for everything such as unwarranted chemotherapy, stem cell therapy, or unnecessary diagnostic procedures. Determining what services are covered, and how much of the cost is covered are not easy questions to answer, but promises to deliver better health for less money than our current system. Physicians, by dint of their training, and responsibility to uphold their profession and protect their patients, understand that healthcare is not a mere commodity. If we are to protect what little autonomy we have left, we need to be a part of the discussion which should not be driven solely by those in the insurance, the hospital and the pharmaceutical industries.

Richard A. Robbins, MD¹

Angela C. Wang, MD²

¹Phoenix Pulmonary and Critical Care Research and Education Foundation, Gilbert, AZ USA

²Scripps Clinic Torrey Pines, La Jolla, CA USA

References

1. Packer M. Medicare for All: Would Patients and Physicians Benefit or Lose? Medpage Today. July 10, 2019. Available at: https://www.medpagetoday.com/blogs/revolutionandrevelation/80926?xid=nl_mpt_blog2019-07-10&eun=g1127723d0r&utm_source=Sailthru&utm_medium=email&utm_campaign=Packer_071019&utm_term=NL_Gen_Int_Milton_Packer (accessed 7/10/19).
2. CMS. National Healthcare Expenditure Data. Available at: https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nationalhealthaccountshistorical.html (accessed 7/11/19).
3. Berchick ER, Hood E, Barnett JC. Health Insurance Coverage in the United States: 2017. September 12, 2018. United States Census Bureau Report Number P60-264. Available at: https://www.census.gov/library/publications/2018/demo/p60-264.html (accessed 7/11/19).
4. Gonzales S,  Sawyer  B.  How does infant mortality in the U.S. compare to other countries? Peterson-Kaiser Health System Tracker. July 7, 2017. Available at: https://www.healthsystemtracker.org/chart-collection/infant-mortality-u-s-compare-countries/#item-start (accessed 7/11/19).
5. Gonzales S, Ramirez M, Sawyer B.  How does U.S. life expectancy compare to other countries? Peterson-Kaiser Health System Tracker. April 4, 2019. Available at: https://www.healthsystemtracker.org/chart-collection/u-s-life-expectancy-compare-countries/#item-start (accessed 7/11/19).

Cite as: Robbins RA, Wang AC. Medicare for all-good idea or political death? Southwest J Pulm Crit Care. 2019;19(1):18-20. doi: https://doi.org/10.13175/swjpcc051-19 PDF

CMS Penalizes 758 Hospitals for Safety Incidents

The Centers for Medicare and Medicaid Services (CMS) is penalizing 758 hospitals with higher rates of patient safety incidents, and more than half of those were also fined last year, as reported by Kaiser Health News (1).

Among the hospitals being financially punished are some well-known institutions, including Yale New Haven Hospital, Medstar Washington Hospital Center in DC, Grady Memorial Hospital, Northwestern Memorial Hospital in Chicago, Indiana University Health,  Brigham and Womens Hospital, Tufts Medical Center, University of North Carolina Hospital, the Cleveland Clinic, Hospital of the University of Pennsylvania, Parkland Health and Hospital, and the University of Virginia Medical Center (Complete List of Hospitals Penalized 2016). In the Southwest the list includes Banner University Medical Center in Tucson, Ronald Reagan UCLA Medical Center, Stanford Health Care, Denver Health Medical Center and the University of New Mexico Medical Center (for list of Southwest hospitals see Appendix 1). In total, CMS estimates the penalties will cost hospitals $364 million. Look now if you must, but you might want to read the below before on how to interpret the data.

The penalties, created by the 2010 health law, are the toughest sanctions CMS has taken on hospital safety. Patient safety advocates worry the fines are not large enough to alter hospital behavior and that they only examine a small portion of the types of mistakes that take place. On the other hand, hospitals say the penalties are counterproductive and unfairly levied against places that have made progress in safety but have not caught up to most facilities. They are also bothered that the health law requires CMS to punish a quarter of hospitals each year. CMS plans to add more types of conditions in future years.

I would like to raise two additional concerns. First, is the data accurate? The data is self-reported by the hospitals and previously the accuracy of these self reports has been questioned (2). Are some hospitals being punished for accurately reporting data while others rewarded for lying? I doubt that CMS will be looking too closely since bad data would invalidate their claims that they are improving hospital safety. It seems unlikely that punishing half the Nation's hospitals will do much except encouraging more suspect data.

Second, does the data mean anything? Please do not misconstrue or twist the truth that I am advocating against patient safety. What I am advocating for is meaningful measures. Previous research has suggested that the measures chosen by CMS have no correlation or even a negative correlation with patient outcomes (3,4). In other words, doing well on a safety measure was associated with either no improvement or a negative outcome, in some cases even death. How can this be? Let me draw an analogy of hospital admissions. About 1% of the 35 million or so patients admitted to hospitals in the US die. The death rate is much lower in the population not admitted to the hospital. According to CMS' logic, if we were to reduce admissions by 5% or 1.75 million, 17,500 lives (1% of 1.75 million) would be saved. This is, of course, absurd.

Looking at hospital acquired infections which make up much of CMS' data, CMS' logic appears similar. For example, insertion of urinary catheters, large bore central lines or endotracheal intubation in sick patients is common. The downside is some will develop urinary, line or lung infections as a complication of these insertions. Many of these sick patients will die and many will have line infections. The data is usually reported by saying hospital-acquired infections have decreased saving 50,000 lives and saved $12 billion in care costs (5). However, the truth is that hospital-acquired infections are often either not the cause of death or the final event in a disease process that caused the patient to be admitted to the hospital in the first place. If 50,000 lives are saved that should be reflected in the hospital death rates or a savings on insurance premiums. Neither has been shown to my knowledge.

So look at the data if you must but look with a skeptical eye. Until CMS convincingly demonstrates that the data is accurate and that their incentives decrease in-hospital complications, mortality and costs-the data is suspect. It could be as simple that the hospitals receiving the penalties are those taking care of sicker patients. What this means is that some hospitals, perhaps the ones that need the money the most, will have 1% less CMS reimbursement, which might make care worse rather than better.

Richard A. Robbins, MD

Editor

SWJPCC

References

1. Rau J. Medicare penalizes 758 hospitals for safety incidents, Kaiser Health News. December 10, 2015. Available at: http://khn.org/news/medicare-penalizes-758-hospitals-for-safety-incidents/ (accessed 12/11/15).
2. Robbins RA. The Emperor has no clothes: the accuracy of hospital performance data. Southwest J Pulm Crit Care 2012;5:203-5.
3. Robbins RA, Gerkin RD. Comparisons between Medicare mortality, morbidity, readmission and complications. Southwest J Pulm Crit Care. 2013;6(6):278-86
4. Lee GM, Kleinman K, Soumerai SB, et al. Effect of nonpayment for preventable infections in U.S. hospitals. N Engl J Med. 2012;367(15):1428-37. [CrossRef] [PubMed]
5. Department of Health and Human Services. Efforts to improve patient safety result in 1.3 million fewer patient harms, 50,000 lives saved and $12 billion in health spending avoided. December 2, 2014. Available at: http://www.hhs.gov/about/news/2014/12/02/efforts-improve-patient-safety-result-1-3-million-fewer-patient-harms-50000-lives-saved-and-12-billion-in-health-spending-avoided.html (accessed 12/11/15).

Cite as: Robbins RA. CMS penalizes 758 hospitals for safety incidents. Southwest J Pulm Crit Care. 2015;11(6):269-70. doi: http://dx.doi.org/10.13175/swjpcc153-15 PDF

The Hands of a Healer

The article in this month’s SWJPCC - "Physical Examination in the Intensive Care Unit. Opinions of Physicians at Three Teaching Hospitals" (1), is a fascinating insight to medical practice and how it has changed with the advent of new technology. The study at three large teaching facilities addressed the questions of how often a physical exam was performed in the ICU, what the perceived utility of the physical exam was, who examines patients most,  and an interesting question pertaining to what exactly constitutes a physical exam. Participants were given theoretical scenarios and answered questions pertaining to the role of a physical exam.  Even though the format was a questionnaire and not direct observation, the results support what I see in clinical practice. The results show that the physical exam, at least in the ICU, is not deemed a critical tool in our armamentarium and that reliance on technology has supplanted the traditional exam. One point that has yet to be formally addressed by this or other studies, is actually how often the physical exam changes the clinical course.

Those of us in my generation remember the days when physical exam was paramount. Indeed, when I was in medical school in England, it was essential and when we presented cases, we had to make a differential diagnosis solely based on the history and physical exam, and then, and only then, would we order specific tests. That was about 25 years ago in London. I suspect that many of my colleagues from that era or earlier, had similar experiences. Modern US practice is to use the physical exam, order a battery of tests and imaging, then come up with the diagnosis.  It has not been shown unequivocally that our reliance on modern imaging and labs is necessarily better.

There are still some scenarios that no laboratory test can pick up. Even in pulmonary medicine, we still teach to treat the patient, not the ABG; and the diagnosis of respiratory failure does not require anything other than a look at the patient. Wheezing shows up on no commonly use lab or imaging in the ICU (excluding less commonly used techniques such as measurement of respiratory system resistance using the ventilator’s sensors and algorithms). There is no question that modern testing is more accurate and provides much more information to us than any, even Oslerian levels of clinical examination could. It also leads to work ups for incidentalomas that may have no real relevance. Conversely all of us probably have anecdotal stories of an exam changing the course. For example, I recall the physical exam that picked the cause of the patient’s agitation, an ulcer on the back of a ventilated, heavily sedated patient. This led to less use of benzodiazepines and a focus on pain control perhaps preventing or mitigating the clinical detriments of excess sedative use in the ICU.

Ordering tests and imaging is usually quicker for the MD than doing a physical exam – one can order three CT scans on three patients in less time than it takes to physically go and exam three patients. This is clearly an improved efficiency for the MD’s work load. The question is then whether the improved efficiency for the MD and added information about the patient from the ancillary testing is worth the extra cost. The physical exam is free except insofar as the time it takes and the effect this has on billing, i.e. that it is still a necessary part of the billing matrix.

The nature of what is a physical exam is also changing. Incorporating bedside imaging with ultrasound is no more a stretch than was incorporating the auscultatory findings when the stethoscope was first introduced. Palpation and percussion in this study, were not deemed necessary parts of the physical exam, which is in sharp contrast the traditional teaching. The perception amongst US physicians that physical exam is more utilized outside the US (England being a typical example) may or may not be true. From the results of this particular study, it seems not to be the case, as there was no difference in responses amongst those who had medical school training outside the US. However even currently, it is impossible to progress in England to higher postgraduate training MRCP or FRCP (member or fellow of the Royal College of Physicians) without being grilled on a physical exam (2).

So where then is the correct balance? As the authors point out, the classic physical findings we were taught are usually present in extreme or end stage disease whereas our purportedly better technology now finds these processes earlier in the clinical course. Pure reliance on either the physical exam or the ancillary testing is not likely to be the correct approach. The answer has yet to be ascertained. A study addressing how often the clinical exam changes the course of a patient’s care significantly (however one may define this) has yet to be done. My prediction is that within 20-30 years, the physical exam will be almost never done in clinical practice.

Clement U. Singarajah, MD

Associate Editor

Southwest Journal of Pulmonary and Critical Care

References

1. Vazquez R, Vazquez Guillamet C, Adeel Rishi M, Florindez J, Dhawan PS, Allen SE, Manthous CA, Lighthall G.  Physical examination in the intensive care unit: opinions of physicians at three teaching hospitals. Southwest J Pulm Crit Care. 2015;10(1):34-43. [CrossRef]
2. Royal College of Physicians of the United Kingdom. MRCP(UK) part 2 clinical examination (paces) guide notes for candidates 2014. Available at: http://www.mrcpuk.org/sites/default/files/documents/Candidate%20guide%20notes%202014_1.pdf (accessed 1/6/15).

Reference as: Singarajah CU. The hands of a healer. Southwest J Pulm Crit Care. 2015;10(1):32-3. doi: http://dx.doi.org/10.13175/swjpcc002-15 PDF

Mild Obstructive Sleep Apnea: Beyond the AHI

Joyce Lee-Iannotti MD

James M Parish MD

Division of Pulmonary Medicine (Dr Parish) Center for Sleep Medicine Department of Neurology (Dr Lee-Iannotti), Center for Sleep Medicine

Mayo Clinic Arizona

Scottsdale, Arizona

A common conundrum faced by sleep medicine practitioners is how to manage the large group of patients with mild sleep apnea. Many patients are referred for sleep evaluation, with symptoms thought to be due to obstructive sleep apnea (OSA). Often polysomnography demonstrates only mild sleep apnea, and the clinician and patient are faced with the dilemma of whether to use continuous positive airway pressure (CPAP) therapy or an oral appliance. In making this important decision the clinician incorporates the commonly used definition of mild sleep apnea as an apnea-hypopnea index of between 5 and 14 apneas or hypopneas per hour of sleep.  Moderate sleep apnea is defined as 15-29 events per hour, and severe is 30 and above events per hour. These arbitrary thresholds originated in the early 1980s when knowledge of this condition was in its infancy and little was known about the long term health effects. The definition was based on the finding of apneas, defined by the complete cessation of airflow for at least 10 seconds. The concept of hypopnea and respiratory-effort related arousal (RERA) came later and with frequently changing definitions that have been the subject of significant controversy throughout the last 30 years.  Many sleep centers include these RERA’s in the definition of respiratory disturbance index, which is incorrectly used interchangeably with AHI. While the sleep literature has demonstrated the untoward effects of moderate to severe sleep apnea, there has been considerable debate about the clinical significance of mild sleep apnea, that is, an AHI between 5 and 15.

The current paper by Quan, et al (1) is a significant contribution to the literature in sleep medicine addressing this important clinical question. This paper reports data drawn from the APPLES study, a large multi-center, well-conducted study designed to determine if CPAP therapy improves sleepiness, mood disorder, or cognitive function in patients with OSA, that has subsequently produced several important publications (2-6). As part of the study, extensive data was obtained on each of these neurocognitive parameters including the Epworth Sleepiness Scale, Stanford Sleepiness Scale, Hamilton Rating Scale for Depression, Profile of Mood States, and Sleep Apnea Quality of Life Index, all validated questionnaires used frequently in the sleep literature. In this part of the study, 199 patients with an AHI>5 but <15 were compared to 40 patients enrolled in the study, but with and AHI<5. The mean AHI was 10 per hour in the mild OSA group, and was 3 per hour in the non-OSA group.  Size of the study was statistically large enough to determine significant differences. Remarkably, there was no significant difference in any rating of sleepiness, mood, or quality of life between the two groups. This study produces an important challenge to the traditional thresholds of disease severity, and raises the question of whether mild sleep apnea based on AHI alone is a disease, and whether it truly requires treatment. Since many patients seen at sleep medicine clinics fall into this category, this is an extremely important question to address.

Several previous studies have attempted to elucidate the issue of mild sleep apnea. Barnes, et al (7) in a randomized controlled trial of CPAP in mild OSA (defined in their study as an AHI 5-30 events per hour) reported that CPAP improved self-reported symptoms of snoring, restless sleep, daytime sleepiness, and irritability, but did not improve objective measure of sleepiness (multiple sleep latency test) or any test of neurobehavioral function, quality of life, mood scores, or 24-hour blood pressure. Weaver, et al (8) reported results from the CATNAP study, a randomized, sham-CPAP controlled study of self reported sleepy patients with mild OSA (defined as AHI 5-30 events per hour) that CPAP significantly improved scores on the Functional Outcomes of Sleep Questionnaire. Both of these trials differ from the current study by defining mild OSA as an AHI up to 30 per hour, whereas the major controversy involves those patients in the AHI 5-15 range. The CATNAP study also selected patients who complained of excessive sleepiness.

The findings from this study emphasize the need to differentiate “obstructive sleep apnea” from “obstructive sleep apnea syndrome.”  Obstructive sleep apnea has been traditionally defined solely by the AHI, whereas OSA syndrome incorporates the subjective and clinical components to the diagnosis (sleepiness, mood disturbance, fatigue, etc.) An abnormal AHI in the mild range without symptoms may not warrant  treatment with CPAP, whereas an excessively sleepy patient with an AHI of 7 would require at least a trial of CPAP with close monitoring. Fatigue, although traditionally associated with mood disorders, is a common symptom in sleep medicine and may be a manifestation of untreated sleep apnea. Future studies could incorporate a fatigue scale (e.g. Fatigue Severity Score) as an adjunct to the Epworth sleepiness score to assess the importance of fatigue as a symptom of OSA.

The current study has an important limitation in that subjects were enrolled based on a referral to a sleep center for some clinical indication related to OSA, and therefore do not represent the general population. It would be possible that individuals drawn randomly from the general population would have lower scores on these tests than a group of subjects referred to a sleep center, which would result in the mild OSA group having significantly different scores on these tests than the general population. In addition the no-OSA group in this study included only 40 patients, and it is possible that a larger group of true no-OSA patients without symptoms causing referral to a sleep center would yield a slightly different result. However, if the untoward effects of mild OSA are indeed significant, it should be relatively easy to find significant abnormalities in mood, sleepiness, and quality of life, and the inability to demonstrate differences in this study group leads one to conclude that the differences, if they exist, are likely to very small.

Besides the mood and quality of life effects of sleep apnea, cardiovascular disease is known to be a significant consequence of obstructive sleep apnea (9).  Stroke, heart failure, myocardial infarction, and atrial fibrillation are known to occur more commonly in untreated OSA than in normal individuals (10). There have been several studies on the cardiovascular effects of mild sleep apnea. The Sleep Heart Health study found a small but significant increase in cardiovascular disease in mild sleep apnea (11).  In another study, Buchner et al (12) found CPAP reduced the risk of subsequent cardiovascular events in patients with mild to moderate (AHI 5-30 per hour) OSA. Therefore, the clinician must look at not only at the AHI, but the larger picture inclusive of presenting symptoms and cardiovascular and cerebrovascular risk factors when deciding on treatment.

Ultimately, this paper challenges the sleep community to look beyond the AHI and improve management algorithms for patients with mild obstructive sleep apnea, with or without symptoms. We propose that an obstructive sleep apnea score be developed, similar to the CHADS-2 score used to determine the need for anticoagulation in patients with non-valvular atrial fibrillation as a means of secondary stroke prevention (13). The “OSA score” could incorporate the AHI, the Epworth sleepiness scale, a quality of life score, a fatigue severity scale, and known cardiovascular and cerebrovascular co-morbidities. A point system could be generated to determine the need for CPAP or alternative therapies.

Hence, this study is likely to be a sentinel study in the sleep medicine literature. Further research in how to “score” patients who need treatment is needed in order to provide best value in management of sleep apnea.

References

1. Quan SF, Budhiraja R, Batool-Anwar S, Gottlieb DJ, Eichling P, Patel S, Wei Shen, Walsh JK, Kushida CA. Lack of impact of mild obstructive sleep apnea on sleepiness, mood and quality of life. Southwest J Pulm Crit Care. 2014;9(1):44-56. [CrossRef]
2. Kushida CA, Nichols DA, Quan SF, et al. The Apnea Positive Pressure Long-term Efficacy Study (APPLES): rationale, design, methods, and procedures. J Clin Sleep Med 2006;2(3):288-300. [PubMed] 
3. Quan SF, Chan CS, Dement WC, et al. The association between obstructive sleep apnea and neurocognitive performance--the Apnea Positive Pressure Long-term Efficacy Study (APPLES). Sleep 2011;34(3):303-14B. [PubMed]
4. Kushida CA, Nichols DA, Holmes TH, et al. Effects of continuous positive airway pressure on neurocognitive function in obstructive sleep apnea patients: The Apnea Positive Pressure Long-term Efficacy Study (APPLES). Sleep 2012;35(12):1593-602. [PubMed]
5. Quan SF, Budhiraja R, Clarke DP, et al. Impact of treatment with continuous positive airway pressure (CPAP) on weight in obstructive sleep apnea. J Clin Sleep Med. 2013;9(10):989-93. [PubMed]
6. Batool-Anwar S, Goodwin JL, Drescher AA, et al. Impact of CPAP on activity patterns and diet in patients with obstructive sleep apnea (OSA). J Clin Sleep Med. 2014;10(5):465-72. [PubMed] 
7. Barnes M, Houston D, Worsnop CJ, et al. A randomized controlled trial of continuous positive airway pressure in mild obstructive sleep apnea. Am J Resp Crit Care Med. 2002;165(6):773-80. [CrossRef] [PubMed] 
8. Weaver TE, Mancini C, Maislin G, et al. Continuous positive airway pressure treatment of sleepy patients with milder obstructive sleep apnea: results of the CPAP Apnea Trial North American Program (CATNAP) randomized clinical trial. Am J Respir Crit Care Med 2012;186(7):677-83. [CrossRef] [PubMed]
9. Newman AB, Nieto FJ, Guidry U, et al. Relation of sleep-disordered breathing to cardiovascular disease risk factors: the Sleep Heart Health Study. Am J Epidemiol. 2001;154(1):50-9. [CrossRef] [PubMed] 
10. Somers VK, White DP, Amin R, et al. Sleep Apnea and Cardiovascular Disease: An American Heart Association/American College of Cardiology Foundation Scientific Statement From the American Heart Association Council for High Blood Pressure Research Professional Education Committee, Council on Clinical Cardiology, Stroke Council, and Council on Cardiovascular Nursing In Collaboration With the National Heart, Lung, and Blood Institute National Center on Sleep Disorders Research (National Institutes of Health). J Am Coll Cardiol. 2008;52(8):686-717. [CrossRef] [PubMed] 
11. Shahar E, Whitney C, Redline S, et al. Sleep-disordered breathing and cardiovascular disease: cross-sectional results of the Sleep Heart Health Study. Am J Respir Crit Care Med 2001;163:19-25. [CrossRef] [PubMed] 
12. Buchner NJ, Sanner BM, Borgel J, Rump LC. Continuous Positive Airway Pressure Treatment of Mild to Moderate Obstructive Sleep Apnea Reduces Cardiovascular Risk. Am J Resp Crit Care Med. 2007;176(12):1274-80. [CrossRef] [PubMed] 
13. Gage BF, Waterman AD, Shannon W, Boechler M, Rich MW, Radford MJ. Validation of clinical classification schemes for predicting stroke: results from the National Registry of Atrial Fibrillation. JAMA. 2001;285(22):2864-70. [CrossRef] [PubMed] 

Reference as: Lee-Iannotti J, Parish JM. Mild obstructive sleep apnea: beyond the AHI. Southwest J Pulm Crit Care. 2014;9(1):40-3. doi: http://dx.doi.org/10.13175/swjpcc099-14 PDF

Multidisciplinary Discussion (MDD) in Interstitial Lung Disease; Some Reflections

Thomas V. Colby MD*

Michael B. Gotway MD^†

Lewis J. Wesselius MD^‡

Departments of Pathology*, Radiology^†, and Pulmonary Medicine^‡

Mayo Clinic Arizona

13400 E. Shea Blvd.

Scottsdale, AZ 85259

Multidisciplinary discussion (MDD) has been used in many disciplines in medicine, notably in thoracic oncology for some two decades (1).  MDD at a multidisciplinary conference (MDC) formalizes activities that have also gone under the label of case conferences, tumor boards, etc. and this practice is time- honored in medical practice.  In the setting of interstitial lung disease (ILD), especially the idiopathic interstitial pneumonias (IIPs) and IPF MDD conducted by a “multidisciplinary team” (MDT) and is now the “gold standard” for diagnosis in this clinical setting (2) and is recommended in the 2011 guidelines for IPF and the 2013 guidelines for IIPs (3, 4). 

Clinical-pathologic correlation, clinical-radiologic-pathologic correlation and clinical-radiologic correlation have been integral to the study of interstitial lung disease since early work of  Heitzman (5), Carrington and Gaensler (6) and many others. This represents the conceptual framework on which the Fleischner Society:  “…an international, multidisciplinary medical society for thoracic radiology, dedicated to the diagnosis and treatment of diseases of the chest” founded in 1969 (7).

The emphasis of MDD in the setting of ILD derives primarily from the study of Flaherty et al (8).  Flaherty et al studied the kappa statistic for intra-observer agreement among expert clinicians evaluating ILD and showed that the kappa significantly improved as more clinical, radiologic and pathologic information was added, suggesting that clinicians had become more confident of their diagnoses with this process. 

In theory, MDD results in a consensus diagnosis based on all the appropriate evidence discussed in a single setting allowing a dynamic intercourse and engagement among the physicians involved.  It allows the physicians to “look each other in the eye” and assess the confidence in the interpretations presented.  It also enables all participating physicians to reassess and change their opinions on the basis of new information and ongoing discussion.  Many of the positive aspects of MDD include the following:

• Dynamic interaction with exchange of ideas
• Engagement of the physicians involved; improved self-esteem
• Physicians can gauge the confidence of others’ opinions/diagnoses (e.g., the radiologic or pathologic diagnosis)
• Physicians can reassess and reinterpret their findings and change their diagnoses
• Educational value for involved physicians (for example, surgeons can appreciate the radiologic findings in terms of where to biopsy; pathologists can appreciate the pathologic findings relative to HRCT)
• Educational value for training fellows and junior staff
• Encourages evidence-based approach
• Increased homogeneity and consistency in managing ILD
• Development of a group ethos with associated improved morale
• Continuous feedback regarding diagnoses
• Forum for developing research ideas
• Forum for discussion and recruitment to clinical trials
• Pooled group clinical experience with broad perspective on ILD (for example, radiologic findings inform the pathologic findings and vice versa)
• An MDD diagnosis might be considered a more defensible diagnosis than individuals’ diagnoses
• The belief that collective thought is better than individuals’ diagnoses

As in any human interaction, theory does not always translate into practice and there are number of issues that  arise with MDD.  Negative and potentially negative aspects of MDD can summarized as follows:

• Physician and allied health staff time
• Physician and allied health staff cost
• Difficulty in coordinating schedules to attend an MDD
• Too many (unselected) cases for discussion
• Lack of a defined protocol and administrative structure for the MDD
• How individual findings should be weighted in terms of final diagnosis
• The effect on the group of individuals’ personalities and stature
• Discourages independence of thought and problem-solving strategies especially for trainees
• Lack of a clear trail as to exactly how a final diagnosis was reached (individual opinions may be lost)
• The “groupthink” phenomenon (to maintain harmony and conformity a group decision may in fact be dysfunctional)
• Over-confidence by the clinician in a diagnosis reached by MDD
• Lack of data on inter- and intra-observer correlation for MDT diagnoses
• When no consensus diagnosis is reached, who is the final arbitrator?
• The phenomenon of “diagnosis drift” (see below)
• The difficulty in validating MDD/MDT diagnoses
• MDD is a luxury of an academic practice and not practical in routine clinical practice
• Medico-legal liability of group members for a group decision

The MDD process for ILD has not been uniformly defined.  Should this be a free-for-all?  Should there be a defined protocol?  The algorithm for the diagnosis of IPF in the 2011 guidelines is a good guide (3).  To some extent, the observations/opinions presented in an MDD are subjective and thus an MDD diagnosis is simply a collection of subjective judgments.  MDD is influenced by individual personalities and there is no question that an “eminence factor” may be at play; a very eminent radiologist may intimidate a relatively inexperienced clinician and the result might be skewed toward the radiologic interpretation.  Cultural factors may also be at play since in some societies age and experience are venerated.  There are no guidelines if a consensus is not reached, and it would be folly to assume that consensus would be reached after every MDD session. When there is no consensus, who is the final arbiter?  We believe the clinician caring for the patient should be the final arbiter. 

Participation in an MDD may leads to something that can be called “diagnosis drift.”  An example of this follows.  The differential diagnosis for IPF includes chronic hypersensitivity pneumonitis, which may show certain radiologic features that suggest that diagnosis.  When such cases are discussed in an MDD, pathologists then become sensitive to similar findings histologically and over time, tend to raise the differential of chronic hypersensitivity pneumonitis more often in the absence any validated confirmation of this practice.

How can MDD be improved?  Given the time, expense, and logistical issues, we think it is unrealistic to expect a MDD for all ILD or IPF cases and that cases for MDD should selected, particularly those where there appears to be discrepancy between the clinical, radiologic and/or pathologic findings. The availability of an electronic medical record (EMR) allows ready access to medical information that may obviate need for MDD in individual cases, although the give and take of discussion is lost.

An attempt should be made to better define the process and the roles of the participants.  We suspect that in most MDDs there is a de facto definition of the process and the roles, but some attempt could be made to formalize this.  Some additional suggestions include:

• Be cognizant of the pros and cons discussed above
• Better defined process with roles and leader clarified
• Preselection of cases to improve efficiency; not all ILD cases need to be discussed
• Include only individuals necessary for a given case (efficient use of staff and their time)
• Consider MDD “overreading” by an experienced group since many community practices will not find MDD to be feasible
• Use of teleconferencing
• Record of the MMD process/decisions
• Continuous reassessment and improvement of the MDD process

And as a final thoughts…..remember that an experienced clinician effectively goes through the process of MDD in the clinical evaluation of an individual patient, appropriately consulting radiologists, pathologists, and other colleagues as needed to reach a management decision……but how is that experience gained…?...The educational value of MDD should not be forgotten.

References

1. Powell HA, Baldwin DR. Multidisciplinary team management in thoracic oncology: more than just a concept? Eur Respir J 2014;43(6):1776-1786. [CrossRef] [PubMed]
2. Wells AU. Histopathologic diagnosis in diffuse lung disease: an ailing gold standard. Am J Respir Crit Care Med 2004;170(8):828-829. [CrossRef] [PubMed] 
3. Raghu G, Collard HR, Egan JJ, Martinez FJ, Behr J, Brown KK et al. An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management. Am J Respir Crit Care Med 2011;183(6):788-824. [CrossRef] [PubMed]
4. Travis WD, Costabel U, Hansell DM, King TE, Jr., Lynch DA, Nicholson AG et al. An official American Thoracic Society/European Respiratory Society statement: Update of the international multidisciplinary classification of the idiopathic interstitial pneumonias. Am J Respir Crit Care Med 2013;188(6):733-748. [CrossRef] [PubMed] 
5. Heitzman ER. The lung: Radiologic-pathologic correlations. Mosby, 1973.
6. Carrington CB, Gaensler EA. Clinical-pathologic approach to diffuse infiltrative lung disease. Monogr Pathol 1978;19:58-87. [PubMed] 
7. Fleischner Society Website. [cited 2014 Jul 1]; Available from: http://fleischner.org/
8. Flaherty KR, King TE, Jr., Raghu G, Lynch JP, 3rd, Colby TV, Travis WD et al. Idiopathic interstitial pneumonia: what is the effect of a multidisciplinary approach to diagnosis? Am J Respir Crit Care Med 2004;170(8):904-910. [CrossRef] [PubMed]

Acknowledgements

The authors thank the Fleischner Society members attending the 2014 Leuven meeting and the following physicians for thoughtful discussion and input:  Jeffrey Galvin, David Hansell, David Lynch, Mathias Prokop, Jay Ryu, and Johny Verschakelen.

Reference as: Colby TV, Gotway MB, Wesselius LJ. Multidisciplinary discussion (MDD) in interstitial lung disease; some reflections. Southwest J Pulm Crit Care. 2014;9(1):32-5. doi: http://dx.doi.org/10.13175/swjpcc097-14 PDF 

VA Administrators Breathe a Sigh of Relief

On May 30, Eric Shinseki, the Secretary for Veterans Affairs (VA), resigned under pressure amidst a growing scandal regarding falsification of patient wait times at nearly 40 VA medical centers. Before leaving office Shinseki fired Sharon Helman, the former hospital director at the Phoenix VA, where the story first broke, along with her deputy and another unnamed administrator. In addition, Susan Bowers, director of VA Veterans Integrated Service Network (VISN) 18 and Helman’s boss, resigned. Robert Petzel, undersecretary for the Veterans Health Administration (VHA, head of the VA hospitals and clinics), had resigned earlier. You could hear the sigh of relief from the VA administrators.

With their bosses resigning left and right, the VA leadership in shambles and the reputation of the VA  soiled for many years to come, why are the VA administrators relieved? The simple answer is that nothing has really changed. There for a moment it looked like real reform might happen. Even President Obama in announcing Shinseki's resignation said the "There is a need for a change in culture..." (1). Shinseki’s resignation would indicate that any action to change the culture is unlikely. Sure a few administrators, like Helman, will lose their jobs, perhaps a few patients will get outsourced to private practioners, but nothing is being done or proposed to change the VA culture. A new interim VA secretary was named and his tenure is likely to be lengthy since no confirmation appears to go unchallenged in the US Congress, and who would want the job?

I was at the VA, when then undersecretary for VHA, Kenneth Kizer, made the fundamental change that resulted in the present mess. Kizer had come to the VA with a program he called the “prescription for change” (2). Indeed, Kizer made several changes but the one that really counted was that the chiefs of staff, doctors who ran the medical services in VA hospitals, were replaced by the head of the Medical Administration Service, usually a business person. This made the VA director the monarch over their own little kingdom, and we all know “it’s good to be the king”. Furthermore, we all know that power corrupts and now with absolute power, the VA director was absolutely corrupted. The hospital directors eliminated any sources of potential opposition. Physicians who did not “play ball” could suddenly find themselves as a target of an investigation (3). After being found guilty by a kangaroo court, their names would be turned over to the National Practioner Databank as bad doctors making it difficult to find a job outside the VA. Those cooperative physicians were rewarded, often for limiting the care of patients. In other words, putting the VA administrators’ interests before the patients’ (4). Lastly, the long-standing relationship with the Nation’s medical schools was destroyed (remember VA dean’s hospitals?). It was argued that the medical schools used the VA to serve their needs. Although this had some truth, it is part of the two-way street that makes cooperation possible. No VA administrator wanted a bunch of doctors and academics telling them what to do.

After eliminating any possible oversight from the physicians or the medical schools, an insulating administrative layer had to be placed between the hospitals and VA central office. Therefore, the Veterans Integrated Service Networks or VISNs, were created. Although ostensibly to improve oversight and efficiency (2), only in Washington would they believe that another layer of bureaucracy would do either. As more and more patients were packed into the system, the numbers of physicians and nurses decreased (5). Not surprisingly, wait times became longer and there was no alternative but to hide the truth. The administrators, the VISNs and VA Central office were all complicit in these lies. Their bonuses depended on it and even when it was discovered by the VA Office of Inspector General (VAOIG) nothing was done.

Congress, who supposedly also provides oversight, was swift to propose action that does not change the VA culture and accomplish little. In this election year Congressional cries to throw those VA bums out have been consistent and loud. However, plenty of clues were available to know that the wait time data was false. First, the concept that you can cut the numbers of physicians and nurses and improve wait times defies common sense. Second, the VAOIG had repeatedly reported that wait times were being falsified. Helman had already been accused of this when she was the director at the Spokane VA (6). This week the Senate passed a bill allowing veterans to see private doctors outside the VA system if they experience long wait times or live more than 40 miles from a VA facility; make it easier to fire VA officials; construct 26 new VA medical facilities and use $500 million in unobligated VA funds to hire additional VA doctors and nurses (7). The VA already is able to do the first two, and as the present crisis illustrates, funds can be diverted away from healthcare. It seems likely this is exactly what will happen unless additional oversight is provided.

Kizer and Ashish Jha authored an editorial on this crisis in the New England Journal of Medicine this week (8). They made three recommendations:

1. The VA should refocus on fewer measures that directly address what is most important to veteran patients and clinicians-especially outcome measures.
2. Some of the resources supporting the central and network office bureaucracies could be redirected to bolster the number of caregivers.
3. The VA needs to engage more with health care organizations and the general public.

All these recommendations are reasonable. Outcome measures, not process of care, should be measured (9). Paying bonuses to administrators for clinicians completing these process of care measures should stop. Many of these measures serve mostly to increase administrative bonuses and not improve patient care. By giving administrators supervisory authority over physicians, healthcare providers were forced to complete a seemingly endless checklists rather than serve the patients' interests.

Bureaucracies should be reduced. VA's central-office staff has grown from about 800 in the late 1990s to nearly 11,000 in 2012 (8). VISN offices have reflected this growth with over 4500 employees in 2012 (10). This diversion of funds away from healthcare is the source of the present problem.

The VA needs to re-engage with the medical schools and with its patients. Reestablishment of the Dean's Committee or other similar system that provides oversight of the VA hospital directors and administrators may be one method of achieving this oversight. The association of the medical schools with the VA served the VA well from the Second World War until the 1990s (11).

Poor pay and micromanagement of physicians to perform meaningless metrics makes primary care onerous. Appropriating funds might improve the salary discrepancy between the VA and the private sector but will not fix the micromanagement problem. The VA may find it difficult to recruit the needed physicians and nurses unless a more friendly work environment is created. How do we know that any appropriated money will be spent on healthcare providers and infrastructure unless additional oversight is put in place? Without oversight the VA positions will become VA vacancies and the VA hospitals will become administrative palaces. Local oversight by VA physicians, nurses and patients is one method of ensuring that appropriated monies are actually spent on healthcare.

VA health care is at a crossroads. New leadership can help the VA succeed but only if the administrative structure is fixed changing the VA culture. Until this occurs the same administrative monarchs will continue to rule their realms and nothing will really change.

Richard A. Robbins, MD*

Editor

Southwest Journal of Pulmonary and Critical Care

References

1. Cohen T, Griffin D, Bronstein S, Black N. Shinseki resigns, but will that improve things at VA hospitals? CNN. May 31, 2014. Available at: http://www.cnn.com/2014/05/30/politics/va-hospitals-shinseki/ (accessed 6/7/14).
2. Kizer KW. Prescription for change. March 1996. Available at: http://www.va.gov/HEALTHPOLICYPLANNING/rxweb.pdf (accessed 6/7/14). 
3. Wagner D. The doctor who launched the VA scandal. Arizona Republic. May 31, 2014. Available at: http://www.azcentral.com/longform/news/arizona/investigations/2014/05/31/va-scandal-whistleblower-sam-foote/9830057/ (accessed 6/7/14).
4. Hsieh P. Three factors that corrupted VA health care and threaten the rest of American medicine. Forbes. May 30, 2014. Available at: http://www.forbes.com/sites/paulhsieh/2014/05/30/three-factors-that-corrupted-va-health-care/ (accessed 6/7/14).
5. Robbins RA. VA administrators gaming the system. Southwest J Pulm Crit Care 2012;4:149-54. Available at: /editorial/2012/5/5/va-administrators-gaming-the-system.html (accessed 6/7/14).
6. Robbins RA. VA scandal widens. Southwest J Pulm Crit Care. 2014;8(5):288-9. Available at: /editorial/2014/5/26/va-scandal-widens.html (accessed 6/7/14). 
7. O'Keefe E. Senators reach bipartisan deal on bill to fix VA. Washington Post. June 5, 2014. Available at: http://www.washingtonpost.com/blogs/post-politics/wp/2014/06/05/senators-reach-bipartisan-deal-on-bill-to-fix-va/ (accessed 6/7/14).
8. Kizer KW, Jha AK. Restoring trust in VA health care. N Engl J Med. 2014 Jun 4. [Epub ahead of print]. Available at: http://www.nejm.org/doi/full/10.1056/NEJMp1406852 (accessed 6/7/14). [CrossRef]
9. Robbins RA, Klotz SA. Quality of care in U.S. hospitals. N Engl J Med. 2005;353(17):1860-1. [CrossRef]
10. VA Office of Inspector General. Audit of management control structures for veterans integrated service network offices. March 27, 2012. Available at: http://www.va.gov/oig/pubs/VAOIG-10-02888-129.pdf (accessed 6/7/14).
11. VA policy memorandum no. 2: policy in association of veterans' hospitals with medical schools. January 30, 1946. Available at: http://www.va.gov/oaa/Archive/PolicyMemo2.pdf (accessed 6/7/14).

*The views expressed are those of the author and do not necessarily reflect the views of the Arizona, New Mexico, Colorado, or California Thoracic Societies or the Mayo Clinic.

Refence as: Robbins RA. VA administrators breathe a sigh of relief. Southwest J Pulm Crit Care. 2014;8(6):336-9. doi: http://dx.doi.org/10.13175/swjpcc077-14 PDF

Why Is It So Difficult to Get Rid of Bad Guidelines?

Reference as: Robbins RA. Why is it so difficult to get rid of bad guidelines? Southwest J Pulm Crit Care 2011;3:141-3. (Click here for a PDF version of the editorial)

My colleagues and I recently published a manuscript in the Southwest Journal of Pulmonary and Critical Care examining compliance with the Joint Commission of Healthcare Organization (Joint Commission, JCAHO) guidelines (1). Compliance with the Joint Commission’s acute myocardial infarction, congestive heart failure, pneumonia and surgical process of care measures had no correlation with traditional outcome measures including mortality rates, morbidity rates, length of stay and readmission rates. In other words, increased compliance with the guidelines was ineffectual at improving patient centered outcomes. Most would agree that ineffectual outcomes are bad. The data was obtained from the Veterans Healthcare Administration Quality and Safety Report and included 485,774 acute medical/surgical discharges in 2009 (2). This data is similar to the Joint Commission’s own data published in 2005 which showed no correlation between guideline compliance and hospital mortality and a number of other publications which have failed to show a correlation with the Joint Commission’s guidelines and patient centered outcomes (3-8). As we pointed out in 2005, the lack of correlation is not surprising since several of the guidelines are not evidence based and improvement in performance has usually been because of increased compliance with these non-evidence based guidelines (1,9).

The above raises the question that if some of the guidelines are not evidence based, and do not seem to have any benefit for patients, why do they persist? We believe that many of the guidelines were formulated with the concept of being easy and cheap to measure and implement, and perhaps more importantly, easy to demonstrate an improvement in compliance. In other words, the guidelines are initiated more to create the perception of an improvement in healthcare, rather than an actual improvement. For example in the pneumonia guidelines, one of the performance measures which have markedly improved is administration of pneumococcal vaccine. Pneumococcal vaccine is easy and cheap to administer once every 5 years to adult patients, despite the evidence that it is ineffective (10). In contrast, it is probably not cheap and certainly not easy to improve pneumonia mortality rates, morbidity rates, length of stay and readmission rates.

To understand why these ineffectual guidelines persist, one needs to understand who benefits from guideline implementation and compliance. First, organizations which formulate the guidelines, such as the Joint Commission, benefit. Implementing a program that the Joint Commission can claim shows an improvement in healthcare is self-serving, but implementing a program which provides no benefit would be politically devastating. At a time when some hospitals are opting out of Joint Commission certification, and when the Joint Commission is under pressure from competing regulatory organizations, the Joint Commission needs to show their programs produce positive results.

Second, programs to ensure compliance with the guidelines directly employ an increasingly large number of personnel within a hospital. At the last VA hospital where I was employed, 26 full time personnel were employed in quality assurance. Since compliance with guidelines to a large extent accounts for their employment, the quality assurance nurses would seem to have little incentive to question whether these guidelines really result in improved healthcare. Rather, their job is to ensure guideline compliance from both hospital employees and nonemployees who practice within the hospital.

Lastly, the administrators within a hospital have several incentives to preserve the guideline status quo. Administrators are often paid bonuses for ensuring guideline compliance. In addition to this direct financial incentive, administrators can often lobby for increases in pay since with the increase number of personnel employed to ensure guideline compliance, the administrators now supervise more employees, an important factor in determining their salary. Furthermore, success in improving compliance, allows administrators to advertise both themselves and their hospital as “outstanding”.

In addition, guidelines allow administrative personnel to direct patient care and indirectly control clinical personnel. Many clinical personnel feel uneasy when confronted with "evidence-based" protocols and guidelines when they are clearly not “evidence-based”. Such discomfort is likely to be more intense when the goals are not simply to recommend a particular approach but to judge failure to comply as evidence of substandard or unsafe care. Reporting a physician or a nurse for substandard care to a licensing board or on a performance evaluation may have devastating consequences.

There appears to be a discrepancy between an “outstanding” hospital as determined by the Joint Commission guidelines and other organizations. Many hospitals which were recognized as top hospitals by US News & World Report, HealthGrades Top 50 Hospitals, or Thomson Reuters Top Cardiovascular Hospitals were not included in the Joint Commission list. Absent are the Mayo Clinic, the Cleveland Clinic, Johns Hopkins University, Stanford University Medical Center, and Massachusetts General.  Academic medical centers, for the most part, were noticeably absent. There were no hospitals listed in New York City, none in Baltimore and only one in Chicago. Small community hospitals were overrepresented and large academic medical centers were underrepresented in the report. However, consistent with previous reports, we found that larger predominately urban, academic hospitals had better all cause mortality, surgical mortality and surgical morbidity compared to small, rural hospitals (1).

Despite the above, I support both guidelines and performance measures, but only if they clearly result in improved patient centered outcomes. Formulating guidelines where the only measure of success is compliance with the guideline should be discouraged. We find it particularly disturbing that we can easily find a hospital’s compliance with a Joint Commission guideline but have difficulty finding the hospital’s standardized mortality rates, morbidity rates, length of stay and readmission rates, measures which are meaningful to most patients. The Joint Commission needs to develop better measures to determine hospital performance. Until that time occurs, the “quality” measures need to be viewed as what they are-meaningless measures which do not serve patients but serve those who benefit from their implementation and compliance.

Richard A. Robbins, M.D.

Editor, Southwest Journal of Pulmonary and Critical Care

References

1. Robbins RA, Gerkin R, Singarajah CU. Relationship between the veterans healthcare administration hospital performance measures and outcomes. Southwest J Pulm Crit Care 2011;3:92-133.
2. Available at: http://www.va.gov/health/docs/HospitalReportCard2010.pdf (accessed 9-28-11).
3. Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in U.S. hospitals as reflected by standardized measures, 2002-2004. N Engl J Med. 2005;353:255-64.
4. Werner RM, Bradlow ET. Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 2006;296:2694-702.
5. Peterson ED, Roe MT, Mulgund J, DeLong ER, Lytle BL, Brindis RG, Smith SC Jr, Pollack CV Jr, Newby LK, Harrington RA, Gibler WB, Ohman EM. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912-20.
6. Fonarow GC, Yancy CW, Heywood JT; ADHERE Scientific Advisory Committee, Study Group, and Investigators. Adherence to heart failure quality-of-care indicators in US hospitals: analysis of the ADHERE Registry. Arch Int Med 2005;165:1469-77.
7. Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med 2008;149:29-32.
8. Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM.  Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303:2479-85.
9. Robbins RA, Klotz SA. Quality of care in U.S. hospitals. N Engl J Med. 2005;353:1860-1.
10. Padrnos L, Bui T, Pattee JJ, Whitmore EJ, Iqbal M, Lee S, Singarajah CU, Robbins RA. Analysis of overall level of evidence behind the Institute of Healthcare Improvement ventilator-associated pneumonia guidelines. Southwest J Pulm Crit Care 2011;3:40-8.

The opinions expressed in this editorial are the opinions of the author and not necessarily the opinions of the Southwest Journal of Pulmonary and Critical Care or the Arizona Thoracic Society.