Evidence-Based Medicine for the Modern Age of Artificial Intelligence

Lambert here: When you get to Figure 2 — the tip of the Evidence-Based Medicine iceberg — it’s not an “iceberg.” It’s a fatberg.

By KLG, who has held research and academic positions in three US medical schools since 1995 and is currently Professor of Biochemistry and Associate Dean. He has performed and directed research on protein structure, function, and evolution; cell adhesion and motility; the mechanism of viral fusion proteins; and assembly of the vertebrate heart. He has served on national review panels of both public and private funding agencies, and his research and that of his students has been funded by the American Heart Association, American Cancer Society, and National Institutes of Health.

According to a Perspective (what might be called a “prospective review”) published in Nature Medicine in January 2023 by Dr. Vivek Subbiah of the University of Texas MD Anderson Cancer Center, an irresistible revolution in clinical medicine is finally at hand, or will be very soon, in the form of a new and much improved version of Evidence-Based Medicine (EBM):

“Recently, advances in wearable technologies, data science and machine learning have begun to transform evidence-based medicine, offering a tantalizing glimpse into a future of next-generation ‘deep’ medicine… The last 30 years have witnessed breathtaking, unparalleled advancements in scientific research – from a better understanding of the pathophysiology of basic disease processes and unraveling the cellular machinery at atomic resolution to developing therapies that alter the course and outcome of diseases in all areas of medicine. Moreover, exponential gains in genomics, immunology, proteomics, metabolomics, gut microbiomes, epigenetics, and virology in parallel with big data science, computational biology, and artificial intelligence (AI) have propelled these advances. In addition, the dawn of CRISPR – Cas9 technologies has opened a tantalizing array of opportunities in personalized medicine.”

Breathtaking.  Alas, the rapid translation of these scientific advances from the “lab bench to bedside” has lagged, thwarted by the slow thinking and slow acting that is Evidence-Based Medicine (EBM) as currently practiced.  Clinical research remains outpaced by basic biomedical research.  Drug development and clinical trials continue to be “expensive for all stakeholders, with a high attrition rate for up to two-thirds of drugs that die in “the ‘valley of death’ between bench and bedside.”  All these bad things plus “the inherent inefficiencies and deficiencies that plague the healthcare system is leading to a crisis in clinical research.  Therefore, innovative strategies are needed to engage patients and generate the necessary evidence to propel new advances into the clinic, so that they may improve public health (and) traditional clinical research should make way for avant-garde ideas and trial designs.”  Breathtaking again, particularly the “avant-garde.”

Dr. Subbiah’s manifesto covers a lot of ground, and there is a lot to unpack here.  However, it would be ridiculous to deny the power and reach of modern biomedical research and EBM when properly conducted, so several of its successes highlighted in this Perspective will be considered here first.

Rare diseases affect several hundred million people worldwide.  Many of these are driven by “small genetically defined or biomarker-associated subsets.”  Thus, they are often targets that suggest a simple intervention, including inhibition of the aberrant protein to manage the disease.  For example, the drug alpelsib [1] [Vijoice, Novartis] inhibits the product of the PIK3CA gene [2], which encodes a subunit of (terminology alert) phosphatidyl inositol 3-kinase.  PI3-kinase is involved in the transmission of signals from outside cells to the inside in pathways that lead to cell proliferation and differentiation.  Thus, PIK3CA-related overgrowth spectrum might be treated by inhibition of PI3-kinase [3].  More importantly for EBM, this was determined/confirmed in a simple single-arm clinical study rather than a large, time-consuming randomized controlled clinical (RCT).  PIK3CA mutants are also found in colon, breast, brain, liver, and stomach cancer.  The underlying practical considerations and results for alpelsib are similar to those applicable to the development of the more well-known imatinib (Gleevec, Novartis) which targets dysregulated ABL tyrosine kinase as part of the BCR-ABL fusion protein produced by the Philadelphia chromosome (perhaps too much information but a fascinating story of basic biomedical research that led to improved clinical practice nonetheless; plus Janet Rowley is too often forgotten).

Non-small cell lung cancer (NSCLC) is generally less sensitive to chemotherapy than other lung cancers.  As it has been characterized at the molecular and cellular levels, NSCLC has been divided into distinct oncogene [4] driven subsets.  This means that clinical trials can use what are known as “synthetic control arms” from real world data (RWD) on previous lung cancer patients rather than a large, randomized cohort in a typical, and slower, randomized controlled trial.  RET fusions are mutations in another kinase that drive some cases of NSCLC.  Treatment with pralsetinib was shown to improve outcomes in RET-fusion positive NSCLC.  Neurofibromatosis 1 (NF1) patients develop tumors that can be debilitating.  Inoperable lesions can be treated with selumetinib which inhibits MEK (still another kinase ).  This was confirmed using a dataset from only 50 pediatric patients, using a single-arm trial showing promising responses to the drug.  Synthetic control arms were initially controversial, but in these cases, where the molecular cause of a rare disease is known, they work well.  But are they a more general model going forward?

Which brings us back to the nature of EBM and how big data science, computational biology, and, above all, artificial intelligence are set to revolutionize the practice of clinical medicine and shrink the duration of the “bench to bedside” period to insignificance.   So, what is EBM?  The definition is somewhat slippery, but here is a lightly edited description from Johns Hopkins Medicine:

EBM is the integration of best research evidence with clinical expertise and patient values…an interdisciplinary approach that uses techniques from science, engineering, biostatistics and epidemiology, such as meta-analysis, decision analysis, risk-benefit analysis, and RCTs to deliver “the right care at the right time to the right patient.”  EBM aims for the ideal that healthcare professionals should make “conscientious, explicit, and judicious use of current best evidence” in their everyday practice.  The practice of evidence-based medicine uses systematic reviews of the medical literature to evaluate the best evidence on specific clinical topics (evidence synthesis). The evidence is then translated…for specific cases using the best research, patient preferences and individual patient characteristics.

Or: Basic biomedical research that elucidates the pathways of normal and abnormal biochemistry, genetics, and physiology identifies promising pathways to treat or manage a disease or condition.  These are then tested for their validity in an RCT.  Nevertheless, it is important to remember that each component of this “three-legged stool” animated and stabilized by research, mechanism, and treatment/clinical medicine based on EBM is just as important as the others; one support cannot be emphasized at the neglect of the others.  This been covered here before in an essay-review of The Illusion of Evidence-Based Medicine by Jon Jureidini and Leemon B. McHenry (2020; the authors later presented their argument in a short, accessible review in BMJ in March 2022).

Nevertheless, EBM exists and has been absolutely essential to progress in clinical medicine despite its failings.  The Salk and then the Sabin polio vaccines were the product of EBM, even it was not as codified then as it is now [5].  A form of EBM identified smoking as the primary cause of lung cancer long before the underlying molecular and cellular mechanisms of cancer were characterized by basic biomedical scientists and physicians.  The same is true for treatments of other cancers that were explained so well in historical context in Emperor of All Maladies by Siddhartha Mukherjee.

Will the revolution outlined in the present article transform EBM into something better, faster, and more complete using artificial intelligence and machine learning, what is referred to as “next-generation deep medicine”?  Perhaps.  But questions arise.  In the words of Dr. Subbiah:

Over the next decade, the application of machine learning, deep neural networks and multimodal biomedical AI is poised to reinvigorate clinical research from all angles, including drug discovery, image interpretation, streamlining electronic health records, improving workflow and, over time, advancing public health (Fig. 1). In addition, innovations in wearables, sensor technology and Internet of Medical Things (IoMT) architectures offer many opportunities (and challenges) to acquire data.

So, let us go to Figure 1. Timeline of drug development from the present to the future.  The take-home message is that in the bad old days (now), from lead compound to approved drug takes as long as 12 years.  But, if we only used AI and machine learning, that would take 2-3 years, tops.  Really?  As I interpret the diagram, leaving out the initial basic research on mechanism (e.g., target identification) the middle section of the new and improved timeline substitutes “next-generation AI technology” for, well, the real work of basic and translational research.  The concepts of networks and hierarchical regulation are very useful in biology, for the good and sufficient reason they have explanatory power and lead to novel hypotheses.  But the nodes in those networks come with labels.  Please forgive me, but as I studied Figure 1 and the next-generation AI/DNN/ML network of unlabeled blue balls in the middle, this famous cartoon from S. Harris came to mind [6].

This is not to say that biomedical science is not “done” faster now than in the very recent past.  Of course, it is.  What took my colleagues and me several years to late-1970s can be done now in weeks with the requisite resources.  My first task in the lab, other than washing glassware, was to purify a protein from 6 kilograms of frozen starting material homogenized in 48 liters of extraction solution.  Eight steps requiring two weeks in a cold room yielded 3-5 milligrams of pure enzyme.  That enzyme was later cloned over a 5-year period from beginning to end.  The end result was that in a few days 100 mg of pure, fully active enzyme can be produced.  A gene cloning sub-project that 15 years ago required a month can now be done in a few days.  Large well-supported academic and NIH labs and those in Big Pharma measure the time in hours.

Has AI made clinical medicine more efficient already?  Perhaps, but that depends on the definition of AI.  Contrary to assertions made in the Perspective, image analysis in radiology, cardiology, gastroenterology, pathology, neurology, dermatology, and ophthalmology is not necessarily AI.  Yes, this is done faster now with improved software and training sets and faster computers than before, but pattern recognition is not AI.  Given that a self-driving car cannot reliably make a left turn in traffic, most of us will still want a pathologist and dermatologist to confirm the diagnosis of basal cell carcinoma and not something worse.

It is certainly plausible that AI will improve how clinical medicine is practiced.  And that would take intelligence, such as that responsible for one of the signal successes of AI in biology, the prediction of a three-dimensional protein structure from the linear sequence of amino acids that make up the protein.  Christian Anfinsen showed in 1961 that the linear sequence of amino acids in the small enzyme ribonuclease was necessary, if not exactly sufficient outside the normal cellular environment, to specify a biological active protein structure.  Sixty years later Alpha-Fold can predict in silico the known structure of RNase, or virtually any other protein of interest, very rapidly.  As someone who listened to many of the early and sometimes heated arguments about protein structure prediction in the late-1990s, this is an achievement of the first magnitude.  It would have made my laboratory much more productive 20 years ago!  Nevertheless, sequence prediction does not have a lot to say about how domains of proteins interact with each other and regulate their mutual functions in cells, tissues, and organisms.  I see no indication that this form of AI can predict how several of my favorite proteins interact with one another to make cell adhesion possible and multicellularity (us) the result.  Maybe this is coming, but the training set of all known protein structures is small compared to the set of interactions of thousands of different proteins, metabolites, and nucleic acid molecules in a congeries of cells and tissues at any given time.  Axiomatic Biology did not work in the in the 1930s, either.  The same consideration applies to Next-Gen EBM: Too many variables, not enough equations, most of which are nonlinear if not outright “stochastic,” to misuse the term somewhat.

This epistemological problem, rather than the solution, is represented in Figure 4: The Evidence-Based Deep Medicine Iceberg.  The message of this Perspective is that the visible part of the floating hunk of multicolored ice will be improved by what lies beneath.  On the one side we find Personalization in the form of EMR (designed for upcoding and billing, not healthcare), wearable technology, and biosensors, all keeping track of our subjects/patients [7] (Surveillance Capitalism, anyone?).  Then “All inputs” (social, behavioral, environmental, genetic) and even more, panomics [8] [genomics, microbiomics and so on].  Nutrition (important and often ignored), patient-reported outcomes (ignored as often as not), research data and all lab data (what’s the real difference?), and something called data fusion.  On the other side of what lies beneath is Deep Learning, informed by natural history and RWD (real world data), which are important for the small clinical trials covered above.  Next are image data, and world medical literature in real time.  Also, research data, once again, and population/epidemiological data, which are also research data.  All medical data is apparently to be subsumed under IoMT – Internet of Medical Things [9].

As a scientist and tutor of medical students, I appreciate the perspective of this Perspective, but this seems to be a long and hypothetical reach.  For example, “All medical data” can be considered little more than a floating signifier.  And one point about the “World medical literature” suffices for the scientific literature: As of 7 February 2023, “COVID” yields 330,982 papers in PubMed in a little over three years (the number went up by ~200 overnight).  I have mentioned this before.  And no, this is still not possible for this factoid to have meaning if the goal of scientific research is to elucidate the mechanisms by which SARS-CoV-2 does its damage.  If, on the other hand, the goal of scientific life is to publish, publish, publish no matter what and where, 8,500 papers per month is business as usual.

But more to the point, Nex-Gen EBM it is highly unlikely to fix EBM, even if the “Big Blue that Lies Beneath” in Figure 4 works.  Yes, the problems with EBM are at some level scientific, which when properly done is not engineering.  Thus, it can be slow.  Important questions often do not readily yield to facile efforts.  And Mother Nature is cryptic; she really does not care what our expectations are.

The biggest problem with EBM as currently conceived is that it is not practiced by disinterested scientists, clinicians, and other healthcare professionals.  Instead, as demonstrated in their complete and abundantly clear presentation, Jureidini and McHenry show that much of EBM is now, and will be still in its next incarnation, an illusion.  Next-Gen EBM remains a subfield of Big Pharma marketing and conventional neoliberal economics.  This is revealed throughout the Perspective, but perhaps especially in one short paragraph:

Before the COVID-19 pandemic, the conduct of clinical research had remained almost unchanged for 30 years and some of the trial conduct norms and rules, although archaic, were unquestioned.  The pandemic exposed many of the inherent systemic limitations in the conduct of trials and forced the clinical trial research enterprise to reevaluate all processes—it has therefore disrupted, catalyzed, and accelerated innovation in this domain.  The lessons learned should help researchers to design and implement next-generation ‘patient-centric’ clinical trials.

One scarcely knows where to begin, so I will simply consider several keywords [10].  Archaic has a connotation, but old does not necessarily mean that norms and rules are outdated, especially when they are properly followed.  And in the common understanding, EBM defines itself: Medicine based on the best available evidence.  The nature of scientific epistemology has been the subject of much disputation, but good science is not difficult to identify.  Neither is bad science.  The pandemic exposed inherent systemic limitations, but they were not the limitations of scientific research and clinical medicine.  Operation Warp Speed did work, but the mRNA vaccines, produced at great cost and greater profit to the exclusion of virtually every other intervention (masks, antivirals, ventilation), prevent neither infection nor transmission of SARS-CoV-2.  Thus, it is passing strange to use the current pandemic as a clarion call to change the clinical research enterprise for the better (“enterprise” is a dead giveaway).  As for disruption, this is indeed a favorite word these days, but that all depends on what is disrupted.

Regarding this, I would note here that “tunable” mRNA vaccines for a rapidly evolving pathogen were a nonstarter from the beginning.  Yes, in the laboratory biologists can produce novel mRNA constructs on a weekly basis to proceed with their research as results indicate, but these are just that, experiments in model systems – mammalian cells, yeast, bacteria, mice.  It will never be proper to experiment on human subjects without transparent clinical trials on the safety and efficacy of each product, i.e., each active component of every novel “tuned vaccine,” with all the data made available for all to see.  No matter how powerful, AI cannot square this circle.  Biology, from bacteria to humans, is much too granular for that.  And while we know much, what we know does not support an adequate training set for what we cannot anticipate.

Finally, while advances in computing power and scientific practice will undoubtedly lead to better interventions and drugs that will cure disease, this will only happen when the setting of Next-Gen EBM leaves the marketing arm of Big Pharma and returns to its scientific roots.  “Next-generation” is a big part of Dr. Subbiah’s Perspective and is a common trope in many fields.  The real solution to our “archaic” EBM has an analogy in a scientific advance that truly was “next generation.”  That would be Next-Gen DNA Sequencing, which is also called massively parallel or deep sequencing.  The details are unimportant, but “massively parallel” is.  The hundreds of millions of people who suffer from rare diseases mentioned above deserve a massively parallel effort on the part of the established biomedical research community to understand the nature and causes of each disease and how each might be treated.  It goes without saying that those of us who suffer from more common but equally devastating conditions deserve the same consideration.

Would some of this research lead to a dead end?  Of course, it would.  And those scientists will move on to the next project with alacrity.  But some of it is likely to lead to unimagined discoveries that could identify alternate pathways of progress.  The world certainly has the infrastructure, scientists, technicians, and healthcare professionals for this project.  And we can afford to do this, period.  All we, as human beings, need is the will.  But all the will in the world pales against powerful antagonists and maladaptive incentives [11].  A solution is up to us.

NOTES

[1]  Helpful hint in parsing the biomedical and clinical literature: A drug ending in the letters “ib” is an inhibitor that binds to a protein (which is frequently an enzyme such as a kinase) and interferes with its activity.  A drug ending in “ab” is a monoclonal antibody that binds to its target and interferes with its function.  Aducanumab comes to mind as an avatar of EBM.

[2]  Another hint: the term PIK3CA in italics represents the gene.  PIK3CA or sometimes PIK2CA3p in regular font represents the protein encoded by the gene, in this case the catalytic (active) subunit of PI-3 kinase.

[3] A kinase is an enzyme that adds a phosphoryl group from ATP onto a target substrate, usually another enzyme or protein.  This modification, called phosphorylation, is reversible and regulates the target, usually by increasing or decreasing activity.  The first oncogene discovered was a kinase: pp60src, “phosphoprotein with a molecular weight of 60,000 that is mutated in cancer caused by Rous Sarcoma Virus.”  Peyton Rous (1879-1970) was awarded the Nobel Prize in Physiology or Medicine in 1966 for his discovery of tumor-inducing viruses, research that dates back to 1910!

[4]  An oncogene is a normal protein that causes cancer when mutated.  Many oncogenes are kinases.

[5]  One might reasonably note that explicit codification of EBM was not necessary in the 1940s and 1950s, or into the 1960s and early-1970s, because the irresistible pressures that vitiate EBM were not ubiquitous then.

[6]  A later version, I think, is here.

[7]  A primary argument of this Perspective places the patient at the center of Next-Gen EBM.  I will leave it to the reader to parse how the patient is at the center of Figure 3.

[8]  “Omic Biology” at one time got on the nerves of old timers, but proteomics, genomics, metabolomics, glycomics (carbohydrates), etc. have all become legitimate demi-disciplines, although many of the new omic-associated journals seem to be too “open-access.”

[9]  The IoMT reference is from a 2018 paper by Menta, Subbiah, and Subbiah “Bringing wearable devices into oncology: fitting smart devices into the clinic” in the journal Discovery Medicine.  270 articles were returned in a search of PubMed on 7 February 2023 with “IoMT” as the query.  It is a thing.  Apparently.

[10]  Keywords help! The locus classicus is here.  Plus more recently here and here.

[11]  From the article: Competing Interests.  None relevant to the manuscript. V.S. reports research funding/grant support for clinical trials from AbbVie, Agensys, Alfasigma, Altum, Amgen, Bayer, BERG Health, Blueprint Medicines, Boston Biomedical, Boston Pharmaceuticals, Celgene, D3 Bio, Dragonfly Therapeutics, Exelixis, Fujifilm, GlaxoSmithKline, Idera Pharmaceuticals, Incyte, Inhibrx, Loxo Oncology, MedImmune, MultiVir, NanoCarrier, National Comprehensive Cancer Network, NCI-CTEP, Northwest Biotherapeutics, Novartis, PharmaMar, Pfizer, Relay Therapeutics, Roche/Genentech, Takeda, Turning Point Therapeutics, UT MD Anderson Cancer Center and Vegenics; travel support from ASCO, ESMO, Helsinn Healthcare, Incyte, Novartis and PharmaMar; consultancy/advisory board participation for Helsinn Healthcare, Jazz Pharmaceuticals, Incyte, Loxo Oncology/Eli Lilly, MedImmune, Novartis, QED Therapeutics, Relay Therapeutics, Daiichi-Sankyo and R-Pharm US; and a relationship with Medscape.