How does one measure hospital quality? Quality occurs along multiple dimensions. Thus, to summarize overall quality, one must create a weighting scheme to compete the distinct quality measures in a single measure. In most cases, quality measures should also account for differences in patient case mix. Hospitals should not be punished with lower quality scores for treating sicker patients. Various risk adjustment methodologies can be applied to account for variation in patient case mix across hospitals.

Today, I review one proposed method for creating composite mortality measures to predict hospital mortality rates after surgery.

### Methodology

**Step 1: Calculate Risk Adjusted Rates for Each Measure**

- Y
_{i}= O_{i}/E_{i}, - O
_{i}= n_{i}^{-1}Σ_{j}y_{ij} - E
_{i}= n_{i}^{-1}Σ_{j}p_{ij}

where *p _{ij}* is the predicted probability based on a logit function [i.e.,

*p*] and

_{ij}=P(y_{ij}=1|X_{ij})*X*are patient covariates such as age, gender, race, and health status. Y

_{ij}_{i}is a

*Kx1*vector of risk-adjusted mortality rate for hospital

*i*.

**Step 2: Calculate Variance-Covariance Matrix**

At the first level, the distribution of the mortality estimate conditional on the structural parameter is:

E(Y_{i} | μ_{i}) = μ_{i} , and Var(Y_{i} | μ_{i}) = V_{i}

where Y_{i} is defined as above and μ_{i} is the corresponding underlying structural quality parameter that represents the average mortality rate that a typical patient could expect at this hospital, and V_{i} is the *KxK* sampling variance for the estimates in Y_{i}.

**Step 3: Conduct Hospital-Level Analysis**

Other factors such as hospital-specific factors such as hospital volume, teaching status, and other factors can also affect expected mortality rates. These variables do not vary by patient; only across hospital. However, one can explicitly incorporate these into a composite measures using the following framework:

- E(μ
_{i}) = Z_{i}β - Var(μ
_{i}) = Ω

where β is a *KxJ* matrix of coefficients capturing the effect of hospital characteristic *j* on patient outcome *k* and Z_{i} is a *Jx1* vector of observable characteristics of hospital *i* that are thought to be related to patient outcomes. Ω is the variance-covariance matrix in the hospital-level quality parameters summarizing the relationships across different dimensions of hospital quality.

One can estimate β using a weighted least squares regression of Y on Z, where one weights by *n _{i}*, the number of patients in hospital

*i*.

One can estimate Ω as follows:

- Ω=N
^{-1}Σ {(Y_{i}) – Z_{i}β)’(Y_{i}) – Z_{i}β) – V_{i}} - Ω=Var(Y
_{i}) – mean(V_{i})

Here, V_{i} is the mean sampling-error covariance matrix. The i-j element of Ω is weighted by the product of *n _{i}* and

*n*. In certain cases, the variance covariance matrix may not be positive-semi definite (e.g., correlations are greater than 1). If this occurs, one can replace the corelation matrix with the nearest positive-semi-definite correlation matrix.

_{j}**Step 4: Bayesian Composite Score**

The estimates Y_{i} are a noisy measures of hospital quality. Further, on must determine how much each measure should contribute to the overall score. Using an empirical Bayesian approach, the weights depend on both the signal (&Sigma) and noise (V_{i}) variances. Thus, the final hospital score is:

- m
_{i}= Y_{i}W_{i}+ Z_{i}β(I-W_{i})

where I is a *KxK* identity matrix and W_{i} is a KxK weighting matrix estimated by:

- W
_{i}= (Ω+V_{i})^{-1}Ω

In other words, the weight is the ratio of the signal variance to the total variance. Thus, estimated composite score places more weight on a hospital’s own outcome (Y_{i}) when the signal is high, but shrinks back toward a conditional mean (Z_{i}β) when the signal ratio is low.

### Attractive Features

- The composite measure incorporate information in a systematic way from many quality measures into a single outcome.
- The parameters are consistently estimated as the number of hospitals increases.
- The estimates maintain many aspects of the existing Bayesian approaches while simplifying the complexity of the estimation.

### Drawbacks

There are some drawbacks to this approach. For instance, this method does not incorporate subjective weights. In the example, a hospital has multiple mortality measures. Since the there is no a priori reason to weight one mortality measure more than another, this approach is useful. However, hospitals may be evaluated based on process of care, patient outcome, and patient satisfaction measures simultaneously. One may wish to place more importance on certain measures than others. However, the approach described above may be useful for calculating domain-level composite scores and then one can aggregate the domain level composites using subjective weights.

### Source

- Dimick, J. B., Staiger, D. O., Osborne, N. H., Nicholas, L. H. and Birkmeyer, J. D. (2012), Composite Measures for Rating Hospital Quality with Major Surgery. Health Services Research, 47: 1861–1879. doi: 10.1111/j.1475-6773.2012.01407.x
- Justin B. Dimick, Douglas O. Staiger, Onur Baser, Zhaohui Fan, John D. Birkmeyer. Composite Measures for Predicting Hospital Mortality with Surgery. Working Paper, 2008.

## Leave a Reply