Econometrics

Risk Adjustment Models

Risk adjustment is important for many aspects of health care.  Medicare uses risk adjustment to modify payments to Medicare Advantage (Part C) plans based on the health of the beneficiaries they cover.  Private insurance companies can use risk adjustment to fine-tune capitation payments to physicians or determine a potential enrollee’s premium.  Providers can use risk adjustment to identify likely high cost patients and how to adjust their likely treatment pattern accordingly.

There is little doubt that risk adjustment is important, but determining which risk adjustment model is ideal is difficult.  A paper by the Society of Actuaries (2007) examines this topic.  The ideal risk adjustment method will depend on a number of factors.   These include:

  • Ease of use of the software;
  • Specificity of the model to the population to which it is being applied;
  • Cost of the software;
  • Transparency of the mechanics and results of the model;
  • Access to data of sufficient quality;
  • Underlying logic or perspective of a model that makes it best for a specific application;
  • Whether the model provides both useful clinical as well as financial information;
  • Whether the model will be used mostly for payment to providers and plans or for underwriting, rating and/or case management;
  • Reliability of the model across settings, over time or with imperfect data (models that are calibrated and tested on a single data set and population may or may not perform well on different data sets/populations);
  • Whether the model is currently in use in the market or organization; and
  • Susceptibility of the model to gaming or upcoding.

The paper evaluates the following models.

  • Adjusted Clinical Groups (ACGs) Version 7.1: incorporates the morbidity-based ACG categories; selected, high-impact, disease-specific Expanded Diagnosis Clusters (EDCs); and diagnostic indicators of the likelihood of future hospitalizations and of being medically frail
  • Chronic Illness and Disability Payment System (CDPS) Version 2.5: This is a diagnosis-based risk assessment model developed by Richard Kronick for use with Medicaid populations.  The model assigns each member to one or more of 67 possible medical condition categories based on diagnosis codes. Each member is also assigned to one of 16 age/gender categories.
  • Clinical Risk Grouping (CRG) Version 1.4. Designed by 3M, the CRG assigns each enrollee to a single risk group based on clinical criteria. The model has  about 1,100 unique groups.
  • Diagnostic Cost Groups (DCGs), RiskSmart Version 2.1.1.  DxGroups are fundamental building blocks of DCG models. All diagnosis codes are grouped into 781 clinically homogeneous groups (DxGroups). These groups are further mapped into 184 hierarchical condition categories. Each patient is also assigned to one of 32 age/gender categories. The model predicts the total medical cost for each patient based upon the HCC and the age/gender category
  • Episode Risk Groups (ERGs) Version 5.3.  Using Episode Treatment Groups (ETG), this model assigns each member to one or more of the 120 possible medical condition categories (called episode risk groups) based on diagnostic and procedural information available on medical and pharmacy claims.
  • Impact Pro.   Developed by Ingenix, The system groups claims into unique episodes of care and other diagnosis-based Impact Clinical Categories (ICCs). These categories describe a member’s observed mix of diseases and conditions and underlying co-morbidities and complications. The ICCs are further grouped into homogenous risk categories (“base-markers”). Each member may be grouped into one or more base-markers and one demographic marker.
  • MEDai.  Individual predictions per member are made using a combination of clinical factors including disease episodes (Symmetry ETGs), drug categories, age, sex, insurance type and other risk markers such as timing and frequency of treatment or diagnosis.
  • MedicaidRx.  Developed by Todd Gilmer at UCSD for use within the Medicaid population, this model assigns each member to one or more of 45 medical condition categories based on the prescription drugs used by each member and to one of 11 age/gender categories. Based on the medical conditions and age/gender categories, the model predicts the overall medical costs for each member. The model includes separate sets of risk weights for adults and children.
  • Pharmacy Risk Groups (PRGs) Version 5.3.  This risk adjustment is based solely on pharmacy prescriptions.  The model maps each NDC to a Pharmacy Risk Group (PRG).  The enrollee’s age, gender and PRG profile determine their risk score.
  • RxGroups, RiskSmart Version 2.1.1.  This is a pharmacy-based risk assessment model developed by researchers and clinicians from Kaiser Permanente, CareGroup of Boston and Harvard Medical School. This model classifies NDCs into 164 mutually exclusive categories (called RxGroups) based on each drug’s therapeutic indication. The model predicts total medical cost based on these RxGroups as well as one of 32 age/gender categories.
  • Underwriting Model, RiskSmart Version 2.1.1.  Used for underwriting employer groups, this model incorporates claim lag into its predictions by providing a six-month lag between the end of the baseline period and the prediction period. The underwriting model uses HCCs, disease interactions, age/gender categories and a prior cost variable to predict future medical costs.

To evaluate the models, the authors use two measures: the mean absolute prediction error (MAPE) and a pseudo-R2 measure.  These are calculated as follows:

  • MAPE = (Σ|Actual-Predicted|) / (Sample Size)
  • R2 = 1 – {[Σ(Actual-Predicted)2] / [Σ(Actual-Average of Actual)2]}

Based on these criteria the authors found the following:

The MEDai methodology included in the study produces the highest R-squared and lowest MAPE among all models. The DCG model produces the highest R-squared and lowest MAPE of the diagnosis input data models. The RxGroups and PRG pharmacy (Pharmacy NDCbased) models generally had good measures, especially considering that they only use pharmacy data. MedicaidRx performs surprisingly well once it is calibrated for the study’s commercial population and a prior cost variable is added, given that it was developed for a Medicaid population. The DxCG Underwriting Model performed well in the underwriting model category (those that include prior costs as inputs in offered model).”

Source:  Winkelman R, Mehmud S (2007) “A Comparative Analysis of Claims-Based Tools for Health Risk Assessment,” Society of Actuaries, April 20, 2007.

1 Comment

Leave a Reply

Your email address will not be published. Required fields are marked *