Unbiased Analysis of Today's Healthcare Issues

Discrete Choice Experiments: Best Practices

Written By: Jason Shafrin - Jul• 11•16

What is the difference between a conjoint analysis and a discrete choice experiement?

ISPOR’s guidelines provide some information.

Conjoint analysisis a broad term that can be used to describe a range of stated-preference methods that have respondents rate, rank, or choose from among a set of experimentally controlled profiles consisting of multiple attributes with varying levels. The most common type of conjoint analysis used in health economics, outcomes research, and health services research (hereafter referred to collectively as outcomes research) is the discrete choice experiment (DCE). The premise of a DCE is that choices among sets of alternative profiles are motivated by differences in the levels of the attributes that define the profiles. By controlling the attribute levels experimentally and asking respondents to make choices among sets of profiles in a series of choice questions, a DCE allows researchers to effectively reverse engineer choice to quantify the impact of changes in attribute levels on choice.

Using a DCE, one can measure the marginal rate of substitution across treatment attributes.  If cost is included as an attribute, one can also estimate respondent willingness to pay for each of the other attributes.  DCEs are limited, however, in that it can only measure preferences across measured attributes.

What are the important DCE components?

The ISPOR guidelines mention ten key components:

  1. research question,
  2. attributes and levels,
  3. construction of tasks,
  4. experimental design,
  5. preference elicitation,
  6. instrument design,
  7. data collection plan,
  8. statistical analyses,
  9. results and conclusions,
  10. study presentation

What statistical analysis is appropriate for DCEs?

First, ISPOR recommends running some basic summary statistics.  You can measure the share of times each attribute was chosen by “counting the number of times each attribute level was chosen by each respondent, summing these totals across all respondents, and dividing this sum by the number of times each attribute level was presented across respondents”

One can use an OLS regression as well where you measure the probability of that a specific profile was chosen and the independent variables are a constant and a vector of attribute levels.  The coefficients can be interpreted as marginal probabilities. “OLS yields unbiased and consistent coefficients and has the advantage of being easy to estimate and interpret. Nevertheless…researchers must assume that the errors with which they measure choices are independent and identically distributed with mean 0 and constant variance.

Most often, however, these data are analyzed using a conditional logit analysis.  In this case, the dependent and independent variables are identical, but one must assume that the residuals follow an independently and identically distributed type 1 extreme-value distribution.  Multinomial logit are sometimes used as well, where multinomial logit typically describes models that relate choices to the characteristics of the respondents making the choices, and  conditional logit relates choices to the elements defining the alternatives among which respondents choose.

The conditional logit requires two key assumptions.

First, the model assumes that choice questions measure utility equally well (or equally poorly) across all respondents and choice tasks (scale heterogeneity). Second, conditional logit does not account for unobserved systematic differences in preferences across respondents (preference heterogeneity).

Another alternative is a mixed logit or random parameters logit model (RPL). This method assumes that “the probability of choosing a profile from a set of alternatives is a function of the attribute levels that characterize the alternatives and a random error term that adjusts for individual-specific variations in preferences.” Whereas conditional logit estimates only the average preference weigth of each attribute across respondents, random logit captures preference heterogeneity by measuring the standard deviation of effects across the sample.

Other statistical approaches include hierarchical Bayes and latent class finite mixture models

How are independent variables represented?

With effects coding, each P value is a measure of thestatistical significance of the difference between the estimatedpreference weight and the mean effect of the attribute. Withdummy-variable coding, each P value is a measure of the statisticalsignificance of the difference between the estimated preference weight and the omitted category.

Researchers must also determine whether to code the variables as continuous or indicator variables. If one uses continuous variables, the analysis has more power. Unlike the indicator variables, however, you cannot identify non-linear relationships if a simple continuous model is used.

How do you measure goodness of fit?

If you use OLS, you can use the standard R-squared metric.  For the conditional logit, however, this statistic is not appropriate.  Instead you can use the log-likelihood metric.  Higher (less negative) values in the log likelihood are associated with a greater ability of a model to explain the pattern of choices in the data.

Also, you can include main effects or interactions. Main effect assumes that attribute preferences are independent of other attributes. For instance, one’s preferences for treatmetn safety may decrease if a treatment is more efficacious. If this is the case, one can model these non-linear preferences using interaction terms.


Children and Cost-Benefit Analysis

Written By: Jason Shafrin - Jul• 10•16

Alan Balch, PhD, discusses the Institute for Clinical and Economic Review’s (ICER’s) Value Assessment Framework tool that aims to provide stakeholders in healthcare with evidence to make more informed decisions regarding new cancer therapies. Dr Balch also considers limitations to the tool’s effectiveness.

Plus, he even discusses why if we lived life using a strict cost-benefit approach, perhaps no one would have kids because the benefits of having kids–like the benefits from health care–are so difficult to monetize.

Weekend Links

Written By: Jason Shafrin - Jul• 09•16

Gates: US Drug pricing system is ‘Better Than Most’

Written By: Jason Shafrin - Jul• 07•16

There has been a lot of criticism of drug prices in the U.S. One person not included the chorus of critics is Bill Gates. In an interview with Bloomberg, he said:

“The current system is better than most other systems one can imagine,” Gates said in an interview on Bloomberg Television. “The drug companies are turning out miracles, and we need their R&D budgets to stay strong. They need to see the opportunity.”

Gates also called for tiered pricing to improve access to innovative drugs for patients in low-income countries. Tomas Philipson, however, says that tiered pricing may not be the best way to help patients in low and middle-income countries in the long-run.

Regarding the economics of innovation into the diseases of the poor, slashing prices of pharma will have the opposite effect on the poor patients they intend to help. It’s well understood that the reason so many individuals with HIV and other prevalent diseases go untreated globally is because there is little money to be made inventing treatments for low-and middle-income countries. For the same reason you would quickly withdraw your money if your pension fund posted negative returns, Wall Street does not invest in global health…Under altruistic care, we should all pay and innovation should stimulated, rather than discouraged, by global health policy. An ideal system has the altruism of rich countries reflected in subsidies for care. These subsidies can then provide rewards, rather than punishments, for innovators that fulfill the world’s altruistic needs.

Quality of Life and Prospect Theory

Written By: Jason Shafrin - Jul• 07•16

Prospect theory states that individuals view transactions relative to a fixed reference point.  Individuals are risk averse for gains (i.e., they would prefer $10 for sure over a 50/50 of winning $0 or $20) but risk loving over losses (i.e., they would prefer a 50/50 ‘lottery’ of losing $0 or $20 over a sure loss of $10.  But are findings for individual preferences for monetary outcomes relevant for patient preferences over quality of life states?

This is the question examined by Attema et al. (2016) using a survey methodology.  They find that:

…health is a fundamentally different commodity than money, with a positive correlation between concavity for gains and for losses. Instead, ‘diminishing sensitivity’ or reflection at the individual level, with a positive correlation between concavity of utility for gains and convexity for losses,is often found in the monetary domain.

Hmmm….that is pretty confusing. Can you give me an example?

For example, people mayevaluate a loss of $11,000 as similar to a loss of $10,000, since the additional loss of $1000 is not perceived as making much of a difference in the context of an already large loss. However, the difference between $1000 and $2000 is perceived as large.

This makes sense.  If you are buying a care, a $500 increase in the price of a $30,000 care seems small, but a friend asking for a $500 loan seems very large.

How is health different?

In the health domain, with 60% as reference point, convexity would mean that a loss from 60% to 40% of QoL would be perceived as much“larger” than a loss from 40% to 20%. We observed the opposite in our experiments: the loss from 60% to 40% was perceived as smaller than the loss from 40% to 20%.


Paging Dr. Watson

Written By: Jason Shafrin - Jul• 06•16

In the future, will your recommended cancer treatment be decided by a computer?  That is what IBM hopes with the launch of their Watson for Genomics project.  CNET reports:

Typically, finding the appropriate treatment for a specific patient means sequencing his or her genome — the complete DNA structure packed into a single cell — finding mutations and then getting a team of seasoned doctors in a room to decide the best options. Watson can do it in less than three minutes…

Watson for Genomics will take on the data-intensive task of searching through sequenced DNA of patients from the Veterans Affairs Department, finding mutations and scanning medical literature to pinpoint the therapeutic treatments that would work best. The program promises to identify customized regimens for 10,000 US veterans during the next two years.

However, patient treatment preferences are likely to vary.  One drug may be more effective but have serious side effects, another may be less effective but fewer side effects.  It is unclear how Watson would take into account these patient preferences.

Thus, a more likely solution is that a Watson-physician team will present treatment options to patients. Physicians will not be replaced, their role will just change.

The pros and cons of retail clinics

Written By: Jason Shafrin - Jul• 05•16

Aaron Caroll does a nice job summarizing these viewpoints in his article in the N.Y. Times’ Upshot:

Researchers for a study published in the American Journal of Medical Quality talked to patients who sought out care at retail clinics. Patients who had a primary care physician, but still went to a retail clinic, did so because their primary care doctors were not available in a timely manner. A quarter of them said that if the retail clinic weren’t available, they’d go to the emergency room.

It’s understandable why physicians’ groups might be opposed to retail clinics. Above and beyond the obvious economic loss when a patient goes elsewhere, many primary care physicians correctly point out that retail clinics often lack the knowledge and experience that come from continuity of care…The American Academy of Pediatrics, the American Academy of Family Physicians and The American Medical Association have all released policy statements or guidelines that oppose, or at least advise, that use of retail clinics be restricted.

Are the doctors right?  On average, the clinics do a pretty good job according to a study by Mehrotra et al. (2009).

Overall costs of care for episodes initiated at retail clinics were substantially lower than those of matched episodes initiated at physician offices, urgent care centers, and emergency departments ($110 vs. $166, $156, and $570, respectively; P < 0.001 for each comparison). Prescription costs were similar in retail clinics, physician offices, and urgent care centers ($21, $21, and $22), as were aggregate quality scores (63.6%, 61.0%, and 62.6%) and patient’s receipt of preventive care (14.5%, 14.2%, and 13.7%) (P > 0.05 vs. retail clinics).

Carroll rightly points out, however, that retail clinics may increase health care cost.  Although cost per visit are lower for retail clinics, they are much more convenient than a doctors visit and are likely to induce more visits.  This is not  a bad thing if patients were forgoing care because physician office’s inconvenient hours were a barrier.

Fourth of July

Written By: Jason Shafrin - Jul• 03•16

Happy Fourth of July weekend.  For Americans, the Fourth of July means celebrating with BBQ, fireworks, friends and family.  But being safe is important as well.

As the U.S. Consumer Product Safety Commission reports, in 2014 there were 11 deaths and more than 10,000 injuries from fireworks that resulted in an ER visit.

Not only is your physical health at risk, but your financial health may also be at risk.  The Connecticut Post reports that:

fireworks affect two different policies – your homeowner’s insurance policy and your health insurance policy. Injuries to yourself or your family are going to be covered by your health care policy. Damages to property and injuries to others will be covered under separate sections of your homeowner’s policy.

This weekend, have fun and be safe.

Friday Links

Written By: Jason Shafrin - Jun• 30•16

After 190 failed tries…

Written By: Jason Shafrin - Jun• 29•16

Why are pharmaceuticals so expensive?  One reasons is that there is a lot of research that goes into developing a drug.  Most of that research results in drugs that don’t work.  One example is the search for Alzheimer’s treatment.  As Bloomberg reports:

Drug companies have long focused on a different protein called amyloid that clumps in the brains of Alzheimer’s patients and is thought to trigger the disease. Companies have poured billions into amyloid-blocking drugs with little success. In 2010, Eli Lilly halted trials of semagacestat after patients on the drug deteriorated more quickly than those on a placebo. In 2012, Pfizer and Johnson & Johnson released results of large trials that showed their amyloid treatment didn’t slow progression of the disease. All told, at least 190 Alzheimer’s drugs have failed in human trials, according to Bernard Munos, a senior fellow at FasterCures, a health nonprofit.

Clearly, Alzheimer’s is one of the most important diseases where treatments have largely failed.  Incentivizing innovation–through higher drug prices, prizes, the patent system, basic research funding, and other avenues–is key to ensure that the fight for effective Alzheimer’s treatments continues.