Academic Articles Information

Quality by any other name

Pay-for-performance or health care quality report cards are the latest fad in medicine. Different types of report cards, however, measure different things. Eve Kerr and co-authors investigate (‘Quality by any other name?’) how different quality measures compare against each other.

The authors look at 3 types of physician review:

  1. Implicit Review: This involves using ‘subjective’ expert opinions regarding whether or not a patient received appropriate care. Most everyone has undergone implicit review; for instance being reviewed by a superior in a subjective fashion is one example of implicit review. Supervisors are better able to measure non-quantifiable quality measures and physicians are less able to ‘game’ the system. On the other hand, the quality measurements are variable depending on who is doing the reviewing.
  2. Focused Explicit Review: These report cards measure whether or not appropriate tasks were completed. Examples of focused review include the Health Plan Employer Data and Information Set (HEDIS), and the VA’s External Peer Review Program (EPRP). The benefit of these focused reviews are that they are concrete, easy to understand, and actionable. One problem is that quality evaluation is limited to certain easily quantifiable dimensions and the system is relatively easy to game.
  3. Global Explicit Review. These systems “use a broader set of quality measures for a much larger number of conditions and the results are reported as summary scores.” The global review is tougher to game and more comprehensive, chart review becomes a more cumbersome process and the review is more expensive to implement. One example of this type of review is the RAND QA Tools system.

The authors conduct a stratified random sample of 26 VA sites of care in the West and Midwest; 621 patients were included in the sample. The authors found that inter-rater reliability was high and that there was high agreement across the 3 types of quality scores. Some important caveats are noted however:

“…it is possible that explicit systems’ process standards do not capture the bulk of important decision making, but providers who do well on aspects of care measured by the explicit tools also do well on aspects of care captured by implicit review but not by the explicit tools. On the other hand, implicit reviewers’ judgment as to what defines a quality problem may have been influenced (and narrowed) by widespread educational efforts focused around existing explicit measures. Finally, we must consider that the performance of explicit systems is dependent on the population. Some process measures may adequately assess quality of care among patients with mild but not severe disease, and agreement between different systems could therefore be dependent on the patient population [my emphasis] considered. “