During my time at US Oncology (USON), I practiced Hematology-Oncology in upstate New York and was a member of the physician leadership team at USON. One of my administrative tasks was to meet with medical directors at commercial health plans to keep them informed of our ongoing projects, as well as to keep our physicians in their good graces. We wanted them to like us. One of the programs we developed at USON was an evidence based clinical decision support tool that promoted value driven prescribing of systemic treatment to patients with cancer. Because the program was designed to provide both quality care as well as generate savings we were certain this would be a real winner when we presented it to the medical directors. I had a meeting with one such medical director of a health plan in upstate New York. I was boasting of how our doctors were practicing evidence based medicine. She sourly replied that she expected all physicians in her health plan’s network to practice evidence based medicine. I was gobsmacked. I thought to myself, “you need to get out more”.
Medicine prides itself on being scientific, yet we refer to the delivery of health care as “medical practice”. I think we can safely say it really is practice. That is not because doctors don’t pay attention to the evidence. But the fact is that the evidence is constantly changing, being updated as scientific knowledge expands. What was a well established fact yesterday may not be one tomorrow. And that pool of scientific knowledge is vast. So doctors cannot possibly keep up with everything that is new. As a result, the care that is delivered is a combination of best evidence, long time practice (experience), and intuition. This is precisely why a standardized approach to clinical decision making is absolutely critical to delivering the best possible care. And ideally that standardized approach collects outcomes and creates an environment in which routine care can lead to improvement in that care (a so-called learning health care system).
Oncology is a perfect use case for a clinical decision support tool. There has been an explosion in the scientific understanding of cancer. Compared to when I was in medical school in the early 1980s we know a tremendous amount about the biology of neoplasia, and this has resulted in great strides in the treatment of cancer. Although we still do not cure many advanced cancers, we treat them far more effectively with fewer side effects. People live longer and they live better. But we should not pretend that this expansion in knowledge has been pain free. And at several points in this journey we were absolutely sure we had it figured out, that there was a “silver bullet”. This of course is not the case, and these “pronouncements” in retrospect, look foolish. For an excellent summary I refer you to the superb Pulitzer prize winning book by Siddhartha Mukherjee, “The Emperor of All Maladies: a Biography of Cancer”.
Since advances have usually been incremental, change has almost always had one foot in the past. The exceptions have been when a truly paradigm shifting discovery was made such as when it was discovered that the over-expression of Her-2 in a subset of breast cancer identified a particularly aggressive subtype. The impact of this was driven home when a treatment, trastuzumab was shown to target this receptor and dramatically improve the outcomes of these breast cancer patients. Evidence based treatment pathways are designed to offer the treatment options appropriate for a given disease as well as ensure that when a marker exists for a unique biological subtype, it is appropriately evaluated and acted upon. All breast cancers need to be tested for the Her-2 overexpression and treated accordingly to optimize patient outcome.
As treatment options for cancer expanded, and as the markers that might predict response to specific treatments increased, the field became challenging if not unmanageable. How could any one physician keep up? Enter the National Comprehensive Cancer Network, the NCCN. In 1993, 13 nationally recognized centers of cancer excellence formed an alliance. The goal was “to ensure delivery of high quality, cost-effective services to people with cancer across the country.” This was to be accomplished by becoming a “developer and promoter of national programs to facilitate the fulfillment of NCCN Member Institution missions in education, research, and patient care”. In 1996, the NCCN released its first clinical practice guidelines for the evaluation and treatment of acute leukemia and breast, colon, lung, ovarian, pediatric, prostate, and rectal cancers.
These guidelines were the product of a panel of experts in each specific cancer type, one expert from each member institution. They were volunteers. A patient advocate was also on each committee. They met on a regularly scheduled basis but also met ad hoc as new evidence emerged. They had a very strict policy on how evidence was to be submitted for consideration and robust firewalls to limit outside influence. Each committee member needed to report potential conflicts of interest. These clinical guidelines were game changers. They were available to physicians free of charge. Ultimately, they expanded to virtually all cancer types. And importantly, they provided the source material for the NCCN compendia.
As I have previously discussed, most of the prescribing in oncology is not based on the FDA label. Rather, payers determine the “appropriateness” of prescribing based on an external review of the evidence which is summarized in compendia. Both Medicare and commercial insurance companies recognize compendia and base coverage policy on them. There have been several recognized compendia over the years, often criticized for the thoroughness and impartiality of their determinations; they are, after all, for profit. When NCCN developed its own compendia everything changed. The NCCN compendia, since it was derived from the NCCN clinical guidelines, had instant legitimacy. And it rapidly became the go-to for health plan coverage policy. It was an absolute god-send to oncologists.
But the NCCN guideline process has not been free of criticism. First, the quality of the evidence that results in NCCN guideline inclusion is not necessarily of the highest standard. It would be ideal if most guideline recommendations were based on randomized clinical trials. But as noted in my discussion of the FDA, in the era of accelerated approval, randomized clinical trials are the exception not the rule. As a consequence, close to 90% of NCCN recommendations are based on consensus of the panel members (NCCN calls this Level 2A). In other words, their opinions based on their experience as subject matter experts as well as the published evidence that exists. I do not in any way wish to denigrate the value of a Level 2A recommendation, but it is opinion (albeit well-informed).
Second, the recommendations are opinion-based and have the potential to be influenced by conflicts of interest. NCCN, as an organization, has a very strict (and strictly enforced) policy. But several publications have pointed out that committee members often have conflicts of interest (https://acsjournals.onlinelibrary.wiley.com/doi/pdf/10.1002/cncr.32997). This is easily explained when one considers that the vast majority of clinical cancer research in this country is funded by the pharmaceutical industry. I personally do not consider this an issue, but it would be a mistake to ignore it. And given how we do clinical research I am not sure how to get around this.
The third criticism is that there is very little consistency across panels. Some panels are “therapeutic enthusiasts”. i.e. they like a lot of what they see. Others are more conservative. The NCCN has decided to let each panel have their own unique personality, often determined by the panel chair. This complicates how recommendations are recognized and implemented by physicians and payers.
Finally, for this discussion, the most important criticism is that NCCN clinical guidelines have been too inclusive. Until fairly recently, they have mostly considered “evidence-based” to permit all appropriate treatments without clear distinction for one treatment as opposed to another. The reason this is an issue is that every oncologist knows all treatments are NOT considered equal in a given disease. Some are more effective. Some are more toxic. Some are appropriate in only specific populations. And some are much, much more expensive than others. While this approach gives doctors (and patients) a lot of choices, that may not always be a good thing.
NCCN has attempted to address this question of value. Their first attempt was called NCCN evidence blocks. These blocks were an attempt to standardize (across committees) how committee members weighed the efficacy, safety, quality of evidence, consistency of evidence, and affordability of treatments. The data was presented in a 5 X 4 matrix (a block), with each domain getting a column in the matrix. A score of five “4s” was the best. Sounds like a plan. But how the committees arrived at their “scores” was not clear. It was totally subjective. In the affordability category, where did they get their cost data? Most physicians have no idea how much things cost and NCCN experts are, for the most part, no different. And given the subjective nature of the “score”, a rating of 4 wasn’t necessarily the same in breast cancer as lung cancer. NCCN evidence blocks never really caught on.
NCCN made a second attempt to tackle the question of value. The committees have developed “Categories of Preference”, designed to articulate what panel experts consider the “best” treatment based on efficacy, toxicity, and cost. In other words, how would the panelist treat their patients. The categories also allow certain treatments that are preferred under certain clinical circumstances, such as if the patient has underlying kidney disease. This approach is much more useful for doctors and patients, but again falls short in that these are very subjective; how cost, for example, is considered is unclear. And by the time you consider all the “circumstances” that permit other treatments you really haven’t excluded many options.
NCCN has done cancer care a great service. They define a standard of care. Their influence is now global. And guidelines have been created for patients as an educational tool and to facilitate shared decision making. But when it comes to the guidelines helping to manage the cost of cancer drugs they definitely fall short. They provide a critical foundation for a value-driven approach, pathways, which is where we will resume our discussion next week.
Keep the great writing Mike! I am re-reading the post because I always learn something new every time!