Open science: any progress yet?

ith the nomination of Jay Bhattacharya for head of NIH and RFK, Jr. for head of HHS, we have a valuable opportunity to modernize how science is conducted in the USA. Both have made valid criticisms of how science is conducted and funded by the government.

Reforms are badly needed. In the past half decade, NIH has turned openly political. The closed publication model has also created a feudalistic system, where the public is told that the details are none of its business until some catastrophe occurs and its cooperation is suddenly needed. At that point, the public is told it must trust everything they're told while being deprived of the actual evidence.

The reformers' goals should be (1) to eliminate all sources of politics from science and (2) to make science more open to the public.

Politics

Politics, regardless of which side it comes from, is inimical to science. It destroys the public's confidence that what scientists produce is the truth and it undermines scientists' confidence that their discoveries will be published and heard.

Politics affects science in three ways.

1. Government is crisis-oriented, so scientific papers and grants routinely portray every scientific problem as a crisis. This is used by government to fund science preferentially on politically fashionable topics. Even if it didn't, any government funding is inherently political, as it encourages dependency on government and thereby incentivizes belief in collectivist ideology that makes government bigger.

2. Increasingly, researchers self-censor to avoid giving “ammunition” to their political opponents. This happens in two ways: biased peer review in controversial fields like climate makes it hard to publish results that conflict with the dogma. Researchers adapt by deliberately misinterpreting their own results to appease the reviewers, claiming that the results support one side while the data clearly support the other. Peer review bias also makes it unlikely that a skeptic will remain in the field, eliminating a valuable “Devil's advocate” that can challenge assumptions and encourage others to be more rigorous.

   The other way is when researchers retract their own papers on the grounds that their results might benefit social positions they consider disreputable. It's a form of pre-emptive censorship that only those who spent a lifetime in the WHSV (wretched hive of scum and villainy) that is academia can fully grasp: if you retract your paper without admitting wrongdoing, you get to smoke out and screw over anybody who was foolish enough to cite you. This is because citing a retracted paper is considered bad scholarship.

   You almost have to admire the diabolical cleverness of this tactic. Write a paper with data that support your opponents' political views; get them to cite you; then retract it, saying it was all a ghastly mistake, leaving your enemies with eggs on their faces.

3. Editors frequently promote their political opinions in their journal. The most egregious example was when the editors of the UK science magazine Nature wrote numerous articles telling Americans how they should vote.

The current closed system has been extensively criticized from all sides. Not only does it deprive researchers in disadvantaged countries from accessing important discoveries, it also creates the problem of “misin­for­mation,” where laymen, unable to read the paper due to the paywall, fill in the missing information with hearsay and misinterpret what has been found. The question is: can such papers, where one academic writes solely for another, be considered real science? Or are they a regression to scholasticism? Can science even survive as a feudal information system? If it tries, it could suffer the same fate as the mainstream media, whose credibility is now almost non-existent.

The Solution

“Open science” is sometimes proposed as a solution, but like all difficult topics there's a tendency to simply re-define the term to fit what we already have. This happened to ‘artificial intelligence’, which now means “something done on a computer,” and ‘consciousness,’ which has so many meanings as to be nearly meaningless.

Redefining the problem is a way of preventing real solutions by side-tracking the discussion, which can be a threat to vested interests. To solve this problem, the real solutions must include:

1. Eliminating government interference
   In biomedicine, funding decisions are made by CSR (Center for Scientific Review), which is part of and subservient to NIH. Despite its stated mission, CSR is becoming more and more political. This is reflected in the increasing percentage of Program Announcements seeking politicized research even in basic science. CSR determines the direction in which research can go, often in direct contradiction to where scientists believe research is needed. The result has been to shift resources away from attempts to identify the pathogenesis of diseases and toward fashionable and even imaginary political causes.

   CSR tries to reduce the problems of hero worship and groupthink that bias grant review, but the organization is hierarchical and undemocratic. Grant funding decisions should be made on the basis of good science, not adherence to fashionable dogma or one person's opinion of where the field should go. Researchers need democratic input into the overall direction for their field. This could be done, for instance, by turning CSR into an independent non-profit, or by repurposing it while preserving its beneficial functions such as grant peer review.

2. Requiring preprints
   Preprints give scientists feedback, which helps them improve their paper, and gives the public a chance to learn where their taxpayer money is going. Currently the public learns about science from popular websites, which generate scientifically inaccurate ‘gee-whiz’ articles based on university press releases written by PR specialists. Or they learn it from activists who misrepresent it. In both cases, the sites promote exaggerated claims that need extensive debunking.

3. Preservation
   The NLM / NCBI's Pubmed Central Initiative (PMC), where science publications are gradually transitioned to an electronic format, is intended to collect and preserve the scientific literature. As traditional print journals fade into history, it's only a matter of time before the only way of accessing scientific knowledge is through websites. This means entire archives could disappear suddenly and without notice.

   Libraries routinely dispose of old printed material. Journals that go bankrupt are rarely accessed and become a cost source. This trend will only accelerate as costs of preservation and storage increase. If science is to survive, the memory of its progress and its missteps must be preserved. PMC seems to be inching toward a policy that any article listed on PubMed must have a free full-text version available. This should be accelerated.

Not all open science initiatives will succeed. ArXiv is the most successful; PLoS is less so. Sites like Biorxiv suffer from the “firehose” problem, where useful information is hard to find in an ocean of irrelevant papers. Even in the traditional literature, it's often said that you can find a paper to support any conclusion you want. This is a problem for researchers on grant review committees: it's easy to get fooled into thinking the idea has support when it doesn't. To reduce the firehose problem, preprint sites like Biorxiv will need to improve their search and cross-referencing software. For instance, if a preprint is later published in a journal, Biorxiv could include a link to the published paper and a record of the reviewers' comments.

Making science more open and less political will also eliminate the causes of so-called “misinformation,” which is another term that is being re-defined into a branch of politics. It will also make plagiarism and data falsification more difficult.

Scientific Fraud and Scientific “Fraud”

A serious risk in the current model is that activists can falsely accuse researchers of data falsification. These claims are a significant contributor to the public's loss of confidence in science. The vast majority are false; typically they come from ambitious individuals in the same institution who have a personal grudge against the investigator. To avoid false claims of falsification, anonymous comments to journals and funding agencies should be rejected without consider­ation of their merit. Those anonymous letters that activists and assistant deans write to sabotage each other's papers and grants must stop.

The rationale for keeping them anonymous is to avoid retribution by the accused. But most accusations are already a form of retribution, perhaps by a jealous colleague, competitor, or activist, so the system favors abuse in one direction and not the other.

One risk of open science is from laymen demanding to see the researcher's raw data. This can mean setting up a server and letting them download all 200 gigabytes of unindexed files with nonsensical filenames like hy22acox1-shsy5y-45l09-1-on-box-wb-2x2-60s-08162022.tif with no information of its meaning or even whether the experiment worked or why it was not published. The layman could misinterpret anything in whatever way he wishes and make wild accusations about the research; or he could turn out to be a competitor who plans to publish your data as his own.

The phenomenon of animal rights activists taking up precious time and releasing expensive animals for political reasons is why animal care facilities are among the most secure buildings in the country, with two-factor access and fingerprint readers. (At one time our lab even considered retinal scans, though we abandoned the idea when one particularly gory spy movie came out.) The history of activists interfering with science is not a happy one. Safeguards are needed for open science.

Readers got a taste of open science during Covid, when it was thought that Covid was so important the public should be informed. The public went wild. They actually read the papers and they discovered the awful truth about medical research: it is in dire need of reform. Research methods, patient selection criteria, and reasoning were often somewhere between flawed and fraudulent; conclusions were often contradicted by the data; alternative interpretations were buried or non-existent; statistics were miscalculated; and important papers that presented unwanted facts were being forcibly retracted while inade­quate­ly based opinions and biased results were not.

By contrast, editorial opinions, which may express thoughtful opinions but often degenerate into rants, can be safely kept behind a paywall. As with political websites, readers typically assume that if it's blocked, it must not be very important, so they won't read it. The general rule in all websites is: if you want people to ignore it, put it behind a paywall. Editors could rant about the US president all they want, and everybody else would just read the science stuff.

The Bureaucracy Is Designed to Destroy Quality

It's sometimes claimed that preprints will reduce the quality of published research. Carneiro et al. ^[1] found that the quality of peer-reviewed articles on Pubmed was only marginally better than on preprint servers on Biorxiv. However, they only studied articles that showed up in both places, which means the two sets of articles were essentially identical. The challenge is that many findings never show up on preprint servers; some that do are simply withdrawn and some others never make it through peer review. Better indexing and search parameters would help, as would allowing unsolicited peer reviews, comments, and revisions.

Funk et al. ^[2], studying the effect of preprints indexed on PMC and Pubmed, also found no difference in quality. So why isn't Biorxiv cited in the literature? The obstacle here is not quality, but academic bureaucrats. A researcher can decide accurately if a finding in his or her specialty is sound, but a bureaucrat cannot. A researcher takes a big risk citing a preprint that might disappear or be retracted at any time—doing so could upset a bureaucrat and cost them a paper or a grant.

Academic bureaucrats routinely terrorize researchers on campus (the story of a professor thrown into a panic because he forgot to put a sticker on something being just one example). NIH rarely commun­ic­ates with the researcher—in fact, usually disallows it—and is happy to let the bureaucrat do whatever he wants. If he decides to cancel a grant out of spite, NIH lets him, and there's no appeal. Nor do they bother to notify the researchers when their grant has been funded.

Government Waste and Corruption

The average person, hearing stories about financial waste, might think the occasional weird project like creating transsexual mice or testing the effect of putting marbles up a cat's butt ^[3] only happens in rare cases. That would mean increased oversight would solve the problem. But it is not so. The demand by universities for more and more government funding has forced researchers to spend over half their time writing grants, leaving little time for research. We end up with angsty articles in science magazines wondering why scientific productivity has declined. Most such articles ignore the possibility that the bureaucratization of academic science funding could be the cause.

Bureaucratization forces researchers to move to “safe” projects, designed not to make discoveries but to remain funded. A “safe project” is one where the outcome is already known or easily predicted, so in fact most current funding amounts to government waste. The often-ridiculed timidity and lack of imagination of academics has its origins in this phenomenon, but it didn't originate with them. It was forced upon them by the US government, and only the US government can abolish it.

As is well known, grants are typically padded by 50% or more over what it costs to do the research. Some goes to electric bills and such, but much of it goes to hiring more bureaucrats. And, as I've mentioned several times on these pages, colleges are quite open about the fact that, unlike in industry, you have to bribe them to hire you. It's not that you need proof of your ability to get a grant. The requirement is for an active grant, which means one that still has money in it that they can take. It's one of many corrupt practices in academia.

Improving the quality of peer-reviewed papers by requiring preprints will make science more open. But the real problem is how government and academic bureaucrats collude to impair the quality of research. Part of the solution must be to repurpose CSR to make it democratic and to change the funding model from bureau­crat-to-bureau­crat to agency-to-researcher to bypass univer­sities' parasitical gate­keepers. People often feel threatened by change, but it would be worth it: any country that did that would soar to the top in scientific discoveries.

dec 23 2024, 7:22 am. last updated jan 01 2025, 5:38 am