Unbiased Analysis of Today's Healthcare Issues

Friday Links

Written By: Jason Shafrin - Nov• 16•17

Is their peer-reviewer bias against research from low-income countries?

Written By: Jason Shafrin - Nov• 15•17

According to a paper from Harris et al. (2017), the country from which a study takes place greatly influences the academic community’s perception of that study.  The authors used a unique study design approach:

In our randomized, controlled, and blinded crossover experiment, participants rated the same abstracts on two separate occasions, one month apart, with the source of these abstracts changing, without their knowledge, between high- and low-income countries.

The high-income countries in the study were the U.S. and Germany and the low-income countries were Ethiopia and Malawi.  Author affiliations also came from high-quality universities within each country [e.g., Harvard University (US), Freiburg University (Germany), University of Addis Ababa (Ethiopia), and University of Mzuzu (Malawi)]. Using this methodology, the authors surveyed 347 English physicians and found that:

…changing the source of a research abstract from a low- to a high-income country significantly improves how it is viewed, all else being equal. Using fixed-effects models, we measured differences in ratings for strength of evidence, relevance, and likelihood of referral to a peer. Having a high-income-country source had a significant overall impact on respondents’ ratings of relevance and recommendation to a peer.

The study detailed results are perhaps more interesting than the top-line findings.  In 3 of 4 sample abstracts, there was no statistically significant difference between the scientific strength of the article depended on where the study took place and the combined results were not statistically significant either.  However, in 3 of 4 cases (and in the combined results) the study in the first-world countries were seen as more relevant; overall peer-reviewers were more likely to recommend papers from high-income counties to their peers.

Thus, it appears there is little scientific bias against the quality of research from low-income countries, but researchers may claim that that research is less relevant to their own or the academic community’s greatest interest.

Interesting throughout.


Are PBM exclusion lists value-based?

Written By: Jason Shafrin - Nov• 14•17

Value-based insurance design (VBID) is a simple concept.  In short, interventions that provide high-value should be covered with little cost sharing; treatments with low-value should be covered with higher rates of cost sharing or in some cases perhaps not even covered at all.

A paper by Cohen et al. (2017) aims to see how far we have come on VBID. Their study asks a more specific question of whether drugsexcluded versus recommended status on pharmacy benefit manager (PBM) exclusion lists corresponds to evidence from cost-effectiveness analyses, lack of evidence, or rebates.  The authors use formulary data from Express Scripts and CVS Caremark.  In particular they examine these two PBM’s use of exclusion lists, which are lists of pharmaceuticals that are excluded from coverage, but are combined with other drugs within the same therapeutic class that the PBM recommends.

In 2016, Express Scripts placed 87 products on its exclusion list, while CVS Caremark put 124 drugs on its exclusion list. This represents a 65 percent increase since 2014. The lists for 2017 indicate CVS has increased the number of excluded products to 154, while Express Scripts have increased its number to 85 (Toich 2017).

The authors aim to examine whether treatments included on the exclusion list had lower estimated coste effectiveness than treatments the were covered (i.e., not on the exclusion list).  The study measures treatment cost effectiveness using data from the Tufts University Cost-Effectiveness Analysis Registry (CEAR).

Using this approach, they find the following:

The mean cost-per-QALY for excluded drugs was higher ($51,611) than the cost-per-QALY for recommended drugs ($49,474), but not statistically significant. We could find no cost-effectiveness evidence in the Registry or peer-reviewed literature for 23 of the excluded drugs, and no evidence for 5 of the recommended drugs.


Quotation of the day: Schumpeter on value

Written By: Jason Shafrin - Nov• 13•17

Nobody values bread according to the quantity of it which is to be found in his country or in the world, but everybody measures the utility of it according to the amount that he has himself…

Schumpeter (1908)

Advancing the Discussion on Real-World Evidence

Written By: Jason Shafrin - Nov• 12•17

With the FDA’s introduction of new guidelines surrounding the use of real-world evidence (RWE) in medical device regulatory decisions, FDA Commissioner Scott Gottlieb advances the argument for the utility of RWE. In fact, the FDA is currently considering the role of RWE in evaluating pharmaceutical treatments. Despite much debate over what part RWE should play in regulatory approval and payer coverage decisions, many of us are left asking two fundamental questions: 1) Why do we need RWE?; and 2) If RWE is needed, how can we use it?

That is the opening paragraph of my article in Drug Discovery and Development last week.  If you want to know the answers to these two questions, do read the whole thing.

Health Wonk Review: Quote-of-the-day Edition

Written By: Jason Shafrin - Nov• 09•17

It’s Health Wonk Review time!  We have some great posts lined up for today’s quotation-themed version of the Health Wonk Review.  We even have a few videos for your viewing pleasure.  Enjoy!

“It’s the economy, stupid” – Bill Clinton


“It is health that is real wealth and not pieces of gold and silver” – Mahatma Gandhi


“Obamacare is dead” – Donald Trump


“America’s health care system is neither healthy, caring, nor a system.” ― Walter Cronkite


“The bigger they are, the harder they fall. And the better the world liked seeing them fall.” –  Loretta Chase



“The man with a new idea is a crank – until the idea succeeds.” – Mark Twain


IVI releases its Open-Source Value Project

Written By: Jason Shafrin - Nov• 08•17

The Innovation and Value Initiative (IVI)–where I serve as the Director of Research–today released its first Open-Source Value Project in rheumatoid arthritis.  I have pasted the press release below.  Go check it out!

Press release: 

The Innovation and Value Initiative (IVI) – a collaboration of academics, patient advocacy organizations, payers, life sciences companies, providers, delivery systems, and other organizations dedicated to finding scientifically credible approaches to measuring value in health care – today launched the IVI Open-Source Value Project, a first-of-its-kind tool to better measure value in health care treatments. The effort kicks off today with the release of an open-source tool focused on measuring value in treatments for rheumatoid arthritis (RA), one of the costliest and potentially debilitating chronic diseases. This open-source tool establishes a platform for the development of several dynamic, transparent and flexible models that will allow any health care stakeholder to scientifically measure value in health care treatments or services.


The Open-Source Value Project is unique in the value assessment space:

  1. Each open-source model is fully transparent. IVI will publicly release research methods, assumptions, results, and external reviews to facilitate the evaluation and replication of its work;
  2. Each open-source model is freely available and usable. IVI will release code, user guides, model documentation and interactive user interfaces. Anyone is free to download the model to run analyses, make changes to the model, build new models with it, or create new apps or tools; and
  3. Each open-source model is developed through a consensus-driven processIVI will ask for input from stakeholders across the health care system to help ensure that the model reflects the different challenges and needs of each particular audience.


“Health care spending has increased dramatically, and health care systems, patients, providers and payers all need better, more transparent tools to measure value in treatments and services,” said Darius Lakdawalla, Executive Director of IVI and Quintiles Chair in Pharmaceutical Development and Regulatory Innovation at the University of Southern California. “The IVI Open-Source Value Project works by creating modeling platforms that are flexible enough to account for different stakeholder perspectives and patient populations – and by launching a transparent and collaborative community process for improving, updating, and expanding the models.”


“The IVI Open-Source Value Project will give payers and health care systems the ability to make clear, data-based decisions about health care spending, provide key insights to patients and providers making shared value-based decisions, and advance the science of value assessment,” added Lakdawalla.


Like the RA model released today, additional robust and flexible platforms for cost-effectiveness modeling are in development for specific diseases. Looking at each disease as a unique clinical entity will allow stakeholders to give more focused input, ultimately providing better evidence and data for how to measure value for each individual disease. Patients were instrumental in the development of the RA model, which is designed to calculate different estimates of value for specific patient characteristics to better capture differences among patients. For example, patient focus groups and reviews by patient researchers helped to define factors in the model that may affect value from a patient’s perspective – such as whether a drug is administered orally versus by infusion. The degree to which the RA model will be able to capture patient heterogeneity will improve as new evidence arises and the model becomes informed by clinical use.


“The debate over how to define and measure value in health care is becoming more vigorous, and divergent views are rooted in different assumptions and perspectives about value and cost effectiveness,” said Dr. Sam Nussbaum, Chair of the IVI Strategic Advisory Panel and strategic advisor to Epstein, Becker, Green (EBG Advisors). “There is an increasing adoption of value-based approaches to health care delivery and services. Almost 30% of the nation’s care delivery is in a value-based arrangement. IVI’s primary goal for the Open-Source Value Project is to deliver consensus-based value models that pave the way for more open and productive discussions about how we measure value in health care and make informed and patient-centered decisions that produce better health outcomes recognizing the need to manage overall health care affordability.”


IVI will regularly update the RA model based on comments and recommendations provided by interested stakeholders, including providers, patient groups, insurers, policy makers and life sciences companies. Comments can be submitted on the IVI website forum, or via e-mail at research@thevalueinitiative.org through January 17, 2018. A panel of experts will consider comments and produce recommendations for changes in the next official IVI version of the model, which will be released in Spring of 2018.


About the Innovation and Value Initiative

The Innovation and Value Initiative (IVI) is a collaboration among thought leaders in academia, patient advocacy organizations, payers, life sciences companies, providers, delivery systems, and other organizations. It was created to raise the level of discussion regarding value in health care and find common ground in the approach to measuring and rewarding value. IVI’s Strategic Advisory Panel includes experienced leaders from across the health care industry. Their role is to provide advice and ensure the credibility and rigor in the research and policy products and to ensure all parties have a voice in IVI activities.

Why don’t we rely exclusively on CEA estimates to make drug coverage determinations?

Written By: Jason Shafrin - Nov• 07•17

Cost effectiveness analyses (CEA) typically analyses the value of a new treatment relative to its cost.  If the value of the additional health benefits outweigh the additional cost, then they should be covered; if the additional cost outweighs the additional benefit, then it should not be covered, right?

There are a few issues with this argument.  First, typical analyses measure cost effectiveness for the average patient.  In many cases, a treatment that is not cost-effective can be cost effective for some patient subgroups but not others.  In theory, one could simply apply the same CEA principles to subgroups, but there are a number of problems with that approach.  First, oftentimes, not all the information about the patient’s likely treatment response–information likey available to the patient and provider–will be available to they payer making coverage decisions.  Secondly, making coverage restrictions (e.g., step therapy, prior authorization) is administratively expensive.

Another factor is that the value of different outcomes may vary by patient.  For instance, working age adults may place a high value on low toxicity and convenience so they can continue working; elderly individuals may focus less on convenience or mode of administration and more on efficacy. These are stereotypes, but they illustrate the point that patients will have different preferences over outcomes whereas QALYs assume that all patients value outcomes the same.

An interesting paper by Espinoza et al. (2017) examines the benefits and costs of centralized compared to individual choice decision-making in a collectively funded system.  Unsurprisingly, they find that:

the value a health care system will have to associate with offering choice to patients for its own sake (independent of its impact on population health) will have to be greater when it is compared with centralized decision‐making based on subgroup recommendations. This result is to be expected because when different decisions for different subgroups have been made, the expected health loss due to lack of understanding of the heterogeneity is lower and the expected population health gains are greater than the average case.

In short, to the extend that central planners can create precise subgroups that reflect patient clinical circumstances and preferences, then choice-based systems are less attractive.  In cases where there is signficiant unobserved clinical or preference-based heterogeneity, however, or in cases where it is expensive to administer treatment choice restrictions, then choice-based systems will be relatively more attractive.

The issue of allowing the maximum amount of patient choice while also managing finite resources is a key question that will require many years of additional research.


IVI Model highlights ACR Conference

Written By: Jason Shafrin - Nov• 06•17

As reported in a Academy of Managed Care & Specialty Pharmacy newsletter, titled “Mathematical Model Aims to Improve Cost-Effectiveness of Rheumatology Treatment“, the Innovation and Value Initiative’s is highlighted from the American College of Rheumatology conference.  Here is an excerpt:

Biological therapies for moderate to severe rheumatoid arthritis have significantly improved treatment and prognosis. However, the high cost of treatment may be a barrier to access. Researchers led by Devin Incerti, a health economist and data scientist at the Innovation and Value Initiative in Oakland, Calif., are developing a cost-effectiveness mathematical model to align prices with value. They shared their latest findings with attendees at the 2017 annual meeting of the American College of Rheumatology (ACR) and the Association of Rheumatology Health Professionals (ARHP).

In their abstract titled, “Open-Source Consensus-Based Models to Improve the Cost-Effectiveness of Rheumatology Care,” the authors said the literature lacks consensus on such models, and that estimates of treatment value vary widely. In addition, they said, because new evidence regarding biologics is being generated rapidly, any model that estimates value must continually evolve and update as well. They called for “transparent, flexible, and accessible cost-effectiveness models that shed light on the implications of different modeling approaches.”

The researchers developed an open-source set of cost-effectiveness models and made it publicly available on GitHub and as an R package. They also created a web application where users can run the model online, modifying their own parameter values and structural assumptions. They then applied a number of different plausible modeling assumptions to their model to compare the cost-effectiveness of a standard sequence of six biologic treatments to conventional disease-modifying antirheumatic drugs (cDMARDs).

Certainly an exciting, new approach to measuring treatment value in RA.

Links to start the week

Written By: Jason Shafrin - Nov• 05•17