Science on Trial, Part 1

Yves here. KLG discusses a new paper which dissects the problems with science, with the investigation of and efforts to prevent and treat Covid as the object lesson.

Since KLG plans to return to this topic soon, I hope he addresses dogmatism, which was aggressively enforced with invocations of The Science and threats (and in some cases successes) in ruining careers of apostates.

By KLG, who has held research and academic positions in three US medical schools since 1995 and is currently Professor of Biochemistry and Associate Dean. He has performed and directed research on protein structure, function, and evolution; cell adhesion and motility; the mechanism of viral fusion proteins; and assembly of the vertebrate heart. He has served on national review panels of both public and private funding agencies, and his research and that of his students has been funded by the American Heart Association, American Cancer Society, and National Institutes of Health.

The current pandemic has exposed various flaws in how scientific research is conducted in the United States and other countries in the so-called Global North.  Some of this can be attributed to the urgency of the task presented by COVID-19 as it emerged four years ago.  Getting a handle on the science of the pandemic is difficult.  The Covid literature is exceedingly large and fragmented and the political response to the pandemic has been worse.  But a paper published online on 8 December 2023 presents a framework for beginning to understand where we are:  The Fragility of Scientific Rigour and Integrity in “Sped up Science”: Research Misconduct, Bias, and Hype in the COVID-19 Pandemic (henceforth Lipworth et al.; the journal website is here; alas, the paper is behind a paywall for most readers).  The authors are from the Department of Philosophy at Macquarie University, the University of Sydney/Royal North Shore Hospital, Sydney Law School, and the University of Toronto.  Although practicing scientists tend to ignore outside voices, this is always a mistake.  This paper is comprehensive, and I look forward to digging deeper into this very useful contribution to the current literature on the exigencies under which modern science, commonly understood, is practiced

As noted in their introduction, “Preclinical and clinical research were sped up and scaled up in both the public and private sectors and in partnerships between them…(resulting) in some extraordinary advances, but it also raised…issues regarding the ethics, rigour, and integrity of scientific research, academic publication, and public communication.”  One may rightly wonder at “extraordinary advances” four years into a pandemic that is still very much a threat, the long-term consequences of which are still unknown but frightening to those who persist in paying attention.  The Spanish Flu pandemic of 1918 that apparently began in Kansas was devastating, killing as many as 50 million people.  But it dissipated after two years, and there was no “Long-Spanish Flu” afterwards.  Recent research may have identified “long-flu” in the relative few who are hospitalized with influenza.

It can be agreed that “these failures were exacerbated by the rush to generate, disseminate, and implement research findings…(creating)…opportunities for unscrupulous actors but also compromised the methodological, peer review, and advisory processes that would usually identify sub-standard research and prevent compromised clinical or policy-level decisions.”  Yes, but these problems are not unique to present circumstances.  They have been baked into the process of Biomedicine for the past forty years, at least.  These problems have been exacerbated by the trajectory of open-access publishing, another part of the story.  Nevertheless, the very reasonable thesis of this paper is that:

Existing strategies for promoting scientific rigour and integrity need to be made more rigourous, better integrated into research training and institutional cultures, and made more sophisticated.  They might also need to be modified or supplemented with other strategies that are fit for purpose not only in public health emergencies but in any research that is sped-up and scaled up to address unmet medical needs.

There can be no argument with this.  But after a short list of apparent dead ends pursued during the pandemic, it will be helpful to dig deeper into what has “gone wrong” with science during COVID-19.  So, what has not worked out?  The usual suspects are on offer: hydroxychloroquine, azithromycin/hydroxychloroquine, remdesivir, convalescent plasma, ivermectin, and molnupiravir, which is no better than a placebo.  Yes, in a rush many of these studies were “confounded by small sample sizes, lack of randomization, missing data” and unsound epidemiology.  Masks and ventilation and other non-pharmaceutical interventions are not considered here to any great extent.  Although it has been largely ignored by the medical establishment, I would also add that the clinical judgment of physicians and healthcare providers in a crisis is also valid and is undoubtedly responsible for much of the reduced burden of COVID-19 mortality and morbidity since the horrific first year of the pandemic.

The “sped-up” scientific literature on COVID-19 is frankly a mess and way too large for any sensible grasp of the disease.  “Everyone” has been in a rush to publish.  As of the evening of 17 December 2023, PubMed contains 396,770 entries containing “COVID” (the search term is case-insensitive) over the past four years.  According to Retraction Watch, 387 (0.098%) of these papers have been retracted along with 18 attached to “expressions of concern.”  So, while the retraction of any scientific paper is a significant event, the COVID literature is not very different from that of any other major topic.  Retractions are very rare, another subject for debate.

The problems with the COVID literature are of another kind, undoubtedly, in the words of Lipworth et al., because “there was more science being conducted more quickly and with less peer and institutional oversight…(and)…a mutually reinforcing and compounding relationship between science and subsequent public health decision-making, regulation and subsidization of interventions…(during which)…commerce and policy act(ed) mutually to erode evidence standards.”  Yes, especially in the current world of “instantaneous communication.” [1]

Has the “speed” of COVID-19 research been the problem, or has the response to the pandemic been “business as usual”?  The pace of biomedical research has certainly increased in the past 50 years, as a consequence of the maturation of molecular biology as a discipline and the development of computational tools to handle terabytes of genomic, physiological, structural, and image data.  Consequently, the imperatives of speed have led to “streamlining” of ethics review in the translation of research findings into medical practice [2]. Of this there can be little doubt.

Lipworth et al. note that even before the pandemic that “training in research methods and research integrity are not always integrated into research training programmes, compliance with institutional research integrity policies can be haphazard, and policies and processes often tend to emphasize governance and risk management more than ethics.”  All true and all covered before, for example here, here, and here.  The question thus arises, “How can scientific rigour, and integrity be understood, motivated, structured, supported and, where necessary enforced in research that is accelerated and scaled up?”

This can be considered from several perspectives, but whether research is “accelerated and scaled up” is fundamentally irrelevant.  Without rigor and integrity, neither science nor scientific research exist.  Moreover, rigor and integrity can only come from the bottom up in how research is conducted, fast or slow, directed or not.  Motivation also comes from the bottom up, whatever the impetus of any research program.  Which is why working, independent, disinterested scientists (there are no other kinds) will find the list of “seven principles to guide global research” from the InterAcademy Partnership [3] both obvious and disheartening:

• Honesty – Conducting research and communication without deception
• Fairness – Treating others with respect and without bias
• Objectivity – Trying to look beyond one’s own conceptions and biases (“trying” is redundant)
• Reliability – Adhering to methods that produce trustworthy results
• Skepticism– Continually reexamining and improving results and explanations
• Accountability – Being willing to justify results and conclusions to other researchers and to society more generally
• Openness – Making data and other information underlying results publicly available

Contrary to the thrust of this very good article, these principles do not “need to be specified in terms of precisely what they mean in the context of sped-up and scaled-up research and publication, what strategies are likely to work for their enactment and enforcement, and who should be responsible for their oversight.”  The lack of any one of these seven principles will destroy any and all scientific research, not only global research on a pressing problem such as COVID-19.  Yes, it is true that they are sometimes “overlooked.” For example, every working scientist in each specialized field knows who tends to be open with results and generous in sharing and who is neither.  Those who publish early and often only tp backtrack later are also well known.  These scientists hurt only their own research programs.

But if principles “used to promote rigour and integrity might also need to be modified or supplemented with other principles and strategies that are fit for purpose in research that is that is funded conducted, reviewed, and disseminated by stakeholders with a wide range of interests and agendas,” then we have ceased talking about scientists and scientific research.  The keyword here is stakeholder, which according to the OED, Second Edition, means (a) one who holds the stake or stakes of a wager; (b) one who has a stake (sense 1c) in something, esp. a business.  (1c) means to have something to gain or lose by the turn of events.  Neither (a) nor (b) has a place in science, although according to Lipworth et al.:

It is important to be realistic about what objectivity actually means and what degree of objectivity is actually possible when commercial actors, consumer [4] advocacy groups, and other political actors not only support research (through financial and in-kind contributions) but also “co-design” and “co-produce” it and assist with “knowledge transfer”…(followed immediately by)…Indeed, one could argue that the goal of “objectivity” should be replaced with the goal of “managed influence and bias” to alert people to the realities of the environment in which they are operating.”

Nail, meet hammer.  And by the way, never forget to read the Acknowledgments/Conflicts of Interest/Funding section of a published paper about anything, but particularly any publication covering a disease or medical condition.

Likewise,  “openness in research might need some re-thinking when research is sped-up and scaled up.”  Yes, it is true that while large independent research groups with unlimited resources might be more likely to be “open,” there is much more to this:

For example, while there is nothing wrong with a company benefiting from a publicly shared data source to develop a product, it is more problematic if they use this publicly accessible informational to develop products that are priced beyond the means of individuals, health systems, or even nations.  It is also problematic if they use the data for purposes that have no public interest…or fail to govern data exchanges in responsible and mutually beneficial ways.  In this way notions such as an “open commons” can be co-opted by commercial or political entities and it is arguable that such entities should be “excluded from the commons and its benefits.”

Further, “existing ideas about openness may also require moderation with it comes to communication of results with the public.”  The kernel of this argument is that scientists faced with the dilemma of needing to encourage behaviors that would mitigate the pandemic (e.g., social distancing, proper masks, vaccination) while not changing their positions too frequently and even “withholding information in order to ensure that their messages were consistent, coherent, and perceived by the public to be valid.”  No, actually.  There is no dilemma here.  The scientists, such as they were, leading the early responses to COVID-19 needed only to remember the Seven Principles, particularly Honesty, Reliability, Skepticism, and Accountability.  This, while also continually reminding everyone, including themselves, that all science is provisional and continuously subject to revision at all times.

Although there were valid arguments that mRNA vaccines against SARS-CoV-2 might not work (e.g., sustained immunity to coronaviruses has been elusive for more than 50 years; mRNA vaccines are obvious, if technically demanding, but have not worked yet – with Zika virus, for example), they were an experiment worth doing under the circumstances.  But these products were an experiment, even if nothing in the techniques used to produce mRNA vaccines was experimental.  Sometimes experiments do not work, no matter how much “sense” they make, as every scientist knows all too well [5]. Would the people have responded differently if the principles of science had been followed during the past four years?  Probably.  I trust the people to respond in kind to trustworthy leaders.  But it has been a while.  Meanwhile the screeching can be deafening.

Which brings us to the following, a proposed addition to the Seven Principles:

The virtue of intellectual humility might need to be added to lists of integrity-related principles because, as science speeds up and scales up, it will be increasingly difficult for collaborators, peer reviewers, and users of research, to fully understand they work they are facilitating, reviewing, and translating into practice.

Actually, no.  Again.  Speed and scale are manageable.  And a legitimate scientist is nothing if not humble, though never lacking in the will to ask serious questions about the natural world.  It is also clear that the failures of science during the pandemic will not disappear when (if) COVID-19 disappears.  And this brings us to the fundamental misunderstanding that I see in Lipworth et al.  Although Biomedical Science is the ostensible subject of this paper, one that all scientists should read and take to heart (and read much of the bibliography, which I have begun), the actual subject here is what I have previously called Biomedicine.  Biomedicine and Biomedical Science are not the same.

Biomedicine comprises Big Pharma, Little Pharma, NIH SBIR/STTR, and the Healthcare System, including PhRMAand AHIP.  You might add the AMA to the mix.  Biomedicine has stakeholders, too many to count.  And this means that when parsing Biomedicine, the  passage quoted above should be modified to “objectivity must be replaced with the goal of understanding outside influence and bias, to alert people to the realities of the environment in which they are placing their trust.” 

Biomedical Science has no stakeholders, other than society writ large.  Biomedical Science is a public good. Biomedicine is not a public good, despite neoliberal protestations and assertions to the contrary.

And therein lies the difference, which suggests a solution if we develop the will (Science on Trial, Part 2, forthcoming).  From Andrew Nelson Lytle of Monteagle: A farm is not a place to grow rich.  A farm is a place to grow corn.  Similarly, a biomedical research laboratory is not a place to “cash out quickly with a unicorn” (PK).  A biomedical research laboratory is a place to reveal (some of) the secrets of Creation. 

Notes

[1] It is significant, however, that since September 1982 “HIV AIDS” (also case-insensitive) returns 168,681 entries over 41 years.  We still have no HIV vaccine, but antiviral treatments do work, in the Global North, and avoidance of HIV is well understood.  There is a long story here, but this one truly will await the historians.

[2] Or another way of saying this is to remember from high school physics that “speed,” which seems to have become paramount, is a scalar quantity of movement in any direction, while velocity is a vector quantity, i.e., with speed plus direction.  Direction has been disregarded, but it can be determinant (e.g., developing an intranasal COVID-19 vaccine that stops SARS-CoV-2 in its tracks or discovering an antiviral drug that prevents viral assembly in those infected and prevents disease) or indeterminant (e.g., characterizing coronaviruses from animals that are likely to transmit the virus to humans).

[3] From the website: “The InterAcademy Partnership (IAP) harnesses the expertise of the world’s leading scientific minds to advance sound policies, improve public health, promote excellence in science education, and achieve other critical development goals.”

[4] “Consumer” is Neoliberal for “citizen.”  I apologize for forgetting who first used this definition in my reading.

[5] This is not the place to argue the success or failure of the mRNA vaccines, but they clearly have not worked as we have come to rightfully expect of vaccines.  They prevent neither transmission nor disease.  The backlash was predictable.  And four years into the pandemic we watch as the politico-scientific establishment makes TV commercials encouraging everyone to get three vaccines for what is now the tripledemic, apparently because COVID-19 has become mostly a minor “seasonal” affliction for a relatively few unlucky people.