Sunday, January 30, 2011
Timeout for Measurement
Meaningful Use Stage 1 is pretty much a done deal with Stage 2 and 3 coming up fast. The Nationwide Health Information Network (NHIN) is slowly coming to life and so is its little cousin Project Direct (formerly known as NHIN Direct). Health Information Exchanges are sprouting up in every state due to Federal largesse or private efforts and according to Dr. Blumenthal the vast majority of hospitals and physicians are on board, or planning to be any day now. The crowds are cheering. First down and ten, at the 50 yard line! Or is it? Would it be too much trouble to ask for a measurement before we move the chains? I am starting to have an uneasy feeling that some adjustments may be necessary if we aim to “win the future”.
The current issue of Archives of Internal Medicine contains a study by Romano & Stafford titled “Electronic Health Records and Clinical Decision Support Systems” (CDS). The study period is from 2005 to 2007 and it is focused on ambulatory visits of over a quarter million patients nationwide. The findings “indicate no consistent association between EHRs and CDS and better quality”. The “rebuttal” comment is quoting the famous 2005 Garg et al. study which reached different conclusions. The Garg review is aggregating studies from the early seventies all the way to 2004 and concludes that “Many CDSSs improve practitioner performance. To date, the effects on patient outcomes remain understudied and, when studied, inconsistent.” I would agree that Romano and Stafford considered rather older technology, but whatever is sold to ambulatory offices today bears almost no resemblance to what Garg et al. studied.
Also this month, a U.K. based study published by the Public Library of Science (PLoS) and examining international publications, including the U.S., from 1997 to 2010, concludes that “There is a large gap between the postulated and empirically demonstrated benefits of eHealth technologies. In addition, there is a lack of robust research on the risks of implementing these technologies and their cost-effectiveness has yet to be demonstrated”. A month earlier, the American Journal of Managed Care published a study funded by the RAND Corporation which examined “the relationship between quality improvement and electronic health record (EHR) adoption in US hospitals”. Using data collected between 2003 and 2007, the authors concluded that “Mixed results suggest that current practices for implementation and use of EHRs have had a limited effect on quality improvement in US hospitals”.
At the very least, it seems that so far EHRs and perhaps even CDS are not making much difference to quality of care. Perhaps we are not measuring the right things. Perhaps all these studies suffer from flawed methodology. Perhaps Meaningful Use certified EHRs are much better than their predecessors. Perhaps when “meaningfully used” EHRs will yield better outcomes. Perhaps our assumptions, and associated expectations, are incorrect. If this is to be a “learning system”, shouldn’t we start learning?
In the January issue of Health Affairs, Advocate Physician Partners of Illinois is describing their model for creating an Accountable Care Organization (ACO), which includes quality improvements and cost reduction. It seems that Advocate had pretty good success with several quality initiatives. Oh yes, Advocate is using lots of HIT and EHRs to collect data, share information, coordinate care and measure progress. A case study for the Rhode Island Quality Institute (RIQI) published by the Commonwealth Fund in December 2010 describes RIQI efforts at improving quality of care in the state. According to the study, RIQI decided in 2005 to reduce ICU complications. By 2009 significant improvements have been achieved and associated costs were reduced by over $3M through a well-planned and massive collaborative effort of all hospitals in the state. There is a nice data table in the report quantifying these improvements. Next, as part of an HIT adoption program, “RIQI chose to increase electronic prescribing by providers as a strategy to reduce prescription errors.” The report goes on to list the impressive eRx adoption rates in Rhode Island, but there is no table for illustrating either reduction in prescription errors or savings associated with such effort. In 2003 Kaiser Permanente embarked on a quality improvement effort dubbed ALL (aspirin, lovastatin and lisinopril) to reduce incidence of heart attacks and strokes in people with diabetes or heart disease. The results as reported in 2009 were very favorable. Somewhere towards the bottom of the report, the Kaiser EHR is mentioned as the means by which patients were identified and collaboratively monitored.
There are many more such examples of quality improvements and cost savings achieved by health systems and physicians with assistance from HIT, with the most recent example of significant savings and health improvements being Dr. Atul Gawande’s latest article in the New Yorker describing cost reductions precisely by providing more and better care to those with highest utilization rates. “Care” in the context of this article refers to sitting with patients in hospital rooms and getting to know them, visiting their homes, assisting them beyond medical care and sometimes speaking to them with a nagging authoritative tone as their mother used to do when they were young. Rather peculiar approach in this day and age where relationships are defined through an Internet browser page. The doctors in the New Yorker article didn’t just stumble upon this archaic way of caring for unfortunate people. They planned, researched and implemented and, yes, they do use computers and data and EHRs. Are we learning yet?
We have reports and studies describing successful quality improvement projects that use HIT as one of the many tools employed by the organization, and we have studies of widespread implementations of HIT, mostly for the sake of implementing HIT, that show no benefits whatsoever. The Meaningful Use project is paying doctors to buy and use HIT in a prescriptive way. There are no stated quality or effectiveness goals, but the architects of Meaningful Use remind us that by 2015, we will be paying physicians for actual outcomes. Outcomes which would be facilitated and measured by all the HIT tools we are now putting in place. So by the time Meaningful Use Stage 3 comes about CMS will be resurrecting a Pay for Performance (P4P) program on steroids. Unfortunately, a new study from the U.K. which tracked half a million patients from 2000 to 2007, concludes that paying doctors for performance does not improve performance, as the chief researcher, Brian Serumaga, told the New York Times, “It seems policy is heading in one direction, while the evidence is heading in another direction”. The most succinct explanation to these findings is provided by an anonymous physician commenting on the study at kevinmd.com: “We are doing the best we can, given the situation we find ourselves in. If you incentivize us, we will still do the best we can given the situation we find ourselves in. Output change: virtually zero.”
We need to change “the situation we find ourselves in” before we can expect any improvements to health care. HIT and EHRs are only as powerful as the people employing them and those people are bound by the situation they find themselves in. The doctors in Rhode Island and those at Advocate and even those working for Kaiser were in a different situation than most. Dr. Jeffrey Brenner featured in the New Yorker story is an outlier who created his own “situation” and is currently operating on the edges of insolvency while saving boatloads of money to the system. There are probably many others out there, but to affect change on a national scale, they need support on a national scale. CMS would be well advised to shift its focus from micromanaging physicians’ tools to creating innovation where it really counts – health care delivery and reimbursement, particularly for Primary Care. Sadly, it’s only fourth down and inches. Do we predictably punt, or do we cleverly adjust a bit and go for it?