Riskgaming

Fabricated Knowledge

Photo by Louis Reed via Unsplash

Finding wisdom within scientific fraud

We produce more knowledge than ever, but our wisdom has never been more lacking.

We produce more research papers than ever, but the speed of new discoveries has never been slower in the modern era.

We produce more content than ever before, and yet, there’s no edifying culture emanating from that work.

Across our knowledge systems, quantity has washed away the sublime quality that we should expect from intellectual works. We’ve transitioned from the definitive paper or essay to counting likes and follows on social media. Our society’s palimpsest of evidence — all those layers of citations and references and metaphors — is increasingly turning to ash as we scorch judiciousness in a conflagration of abundance.

The latest casualty happens to be one that hits very close to home for me. Based on the work of Vanderbilt University physician-neuroscientist Matthew Schrag, Charles Piller in Science published an in-depth investigation a few weeks ago into research that oligomer species Aβ*56 might be the “star suspect” in triggering the onset of Alzheimer’s disease. A series of papers, led by Sylvain Lesné, used laboratory experiments in mice to show that the oligomer was toxic and elucidating the causal mechanism.

The original paper in the series has been cited nearly 2,300 times according to Nature, and it has become the basis for an entire field of scientific endeavor. The suspected oligomer is so promising, in fact, that Piller notes, “Since then, annual NIH support for studies labeled ‘amyloid, oligomer, and Alzheimer’s’ has risen from near zero to $287 million in 2021. Lesné and [Karen Ashe] helped spark that explosion, experts say.”

There’s just one problem: the groundwork under that sequence of papers appears to be an entire fabrication.

Piller writes:

In a PubPeer search for “Alzheimer’s,” postings about articles in The Journal of Neuroscience caught Schrag’s eye. They questioned the authenticity of blots used to differentiate Aβ and similar proteins in mouse brain tissue. Several bands seemed to be duplicated. Using software tools, Schrag confirmed the PubPeer comments and found similar problems with other blots in the same articles. He also found some blot backgrounds that seemed to have been improperly duplicated.

And

Science asked two independent image analysts—[Elisabeth Bik] and Jana Christopher—to review Schrag’s findings about that paper and others by Lesné. They say some supposed manipulation might be digital artifacts that can occur inadvertently during image processing, a possibility Schrag concedes. But Bik found his conclusions compelling and sound. Christopher concurred about the many duplicated images and some markings suggesting cut-and-pasted Western blots flagged by Schrag. She also identified additional dubious blots and backgrounds he had missed.

The images in Science are extraordinarily damning. Really, you should take a look.

Investigations are underway, and the landmark Nature study now has an editor’s note advising readers “to use caution when using results reported therein.”

But let’s step back and consider the scale here: after 16 years and billions of dollars of aggregate government funding, one of the major lines of inquiry in Alzheimer’s research has been found to be based — at minimum — on deeply flawed evidence. This is not a sterling example of scientific self-correction, but rather an embarrassing indictment of a scientific enterprise where even basic facts are never checked for accuracy. Per Piller:

Given those findings, the scarcity of independent confirmation of the Aβ*56 claims seems telling, [Dennis Selkoe] says. “In science, once you publish your data, if it’s not readily replicated, then there is real concern that it’s not correct or true. There’s precious little clearcut evidence that Aβ*56 exists, or if it exists, correlates in a reproducible fashion with features of Alzheimer’s—even in animal models.”

After 16 years and billions of dollars of aggregate government funding, we don’t even really know if the oligomer exists.

My “indictment” isn’t on Lesné (or at least, not exclusively), but rather on an experimental system that is failing to meet our present needs. This is not a culture of wisdom, a culture that is designed to seek and secure fundamental truths. It’s a culture that is designed to grab research headlines and allow for quick-fire publications, where nothing is verified and every groundbreaking paper is potentially quicksand.

We talk a lot about Enlightenment values around here (Lux means light after all), but these critical values are useless without systems that actually produce the kind of rigorous wisdom those values demand. Or, as they say in statistics: garbage in, garbage out.

William Davies, in his book Nervous States, talks about the genesis of the Royal Society, which offered a context where natural philosophers could share knowledge and debate its construction and implications. Now most data is proprietary, including some of the best datasets. Evidence is increasingly unavailable for public argument.

Indeed, Schrag can’t prove fraud at all. As Piller writes, “That would require access to original, complete, unpublished images and in some cases raw numerical data.” That’s data that Schrag — and all of us — don’t have. “Authors sometimes share those with researchers conducting similar work, although they usually ignore such requests, according to recent studies of datasharing practices.”

We’ve reached a nadir in science where not only do we not know whether the oligomer exists, but the modicum of evidence for its existence isn’t even available for scrutiny.

Now, there are a bunch of themes here: the replication crisis in science, the publish-or-perish culture of modern research labs, the social power hierarchies that prevent junior, less-established (and often more skeptical) researchers from questioning the distinguished giants in their fields. But a more important theme is not that science is faulty, but that our entire knowledge system is running dangerously ragged.

There is a solution here, but it requires more than adding additional bureaucratic overhead to an already overstretched system. This week, I got around to reading a thinking classic, A.D. Sertillanges’s The Intellectual Life. Sertillanges was a French Catholic philosopher who wanted to channel the teachings of Saint Thomas Aquinas to a modern audience in 1921 following the devastation of World War I. It also happens to be a strangely modern self-help book for an almost exactly one-century-old treatise, with comprehensive advice on how to read, how to manage a daily schedule, how to write notes, and how to imbibe the ferment of ideas.

Sertillanges argues that solitude is the key to making headway in sciences, literature and other intellectual endeavors. But that solitude should be used to work with the world, not to shut it out:

To stay in one’s study, and to indulge in the interior babble, the solicitations of desire, the exultation of pride, the floodtide of thoughts that introduce within us an absorbing and discordant outside world — would that be solitude? There is a false solitude as there is a false peace. On the contrary, to go out and act through duty, or wisdom, or for the sake of that relaxation of which we shall later plead the necessity, may be a higher kind of solitude, which feeds and tones up the soul instead of depressing and weakening it.

He’s not arguing that intellectuals should become monks or ivory tower hermits, but rather that they should go forth into the world with a deep well of wisdom. There’s an obvious religious overtone, but the point is secular as well: to engage with the world, to comprehend the eddies of content and knowledge that whirl around us, we need to know ourselves first and foremost. We need to pull back to move forward.

Sertillanges’s best advice is the antidote we most need: to cut down on superfluous knowledge and focus our efforts on discovering and understanding fundamental truths. In a section entitled “Not Reading Much”, he writes:

The continual sight stimuli thus occasioned destroys mental energy, as constant vibration wears out steel. There is no real work to be expected from the great reader, when he has overstrained his eyes and the membranes of his brain; he is in a state of chronic mental headache, while the wise worker, preserving his self-control, calm and clear-headed, reads only what he wants to retain, retains only what will be useful, manages his brain prudently and does not abuse it by cramming it absurdly.

We need less scientific papers, but more high-quality and well-vetted ones. We need less breaking news, and more well-reported stories. We need less content, but more works that transform the human mind.

Getting there requires rebuilding our cultural and knowledge institutions to prioritize quality over quantity, to expend much more prodigious effort in building up the best rather than publishing plenty. Less is more, and a lot less is a lot more. We might even find a lot more wisdom along the way.

Building tech+bio communities and the denominator effect

The
The "Securities" podcast. Artwork by Chris Gates

There has been a massive expansion in data emanating from bio labs, and that means next-generation AI algorithms and machine learning models finally have the grist to transform the future trajectories of biology and health therapies. Yet, there’s a key translation challenge: how do you get computer scientists and biologists — two types of specialists with very different training — to collaborate with each other effectively?

Two groups, Bits in Bio and Nucleate, have independently spearheaded new ways of bringing all people interested in tech and biology together to share best practices and think through patterns of startup inception and growth.

In a new “Securities” podcast episode, Michael Retchin, a PhD student at Weill Cornell Medicine and the founder of Nucleate, and Nicholas Larus-Stone, the first software engineering hire at Lux-backed synthetic biology startup Octant.bio as well as the founder of Bits in Bio, join Shaq Vayda and me to talk about where tech + bio (versus “biotech”) is coming from, how the two community leaders launched and grew their respective organizations, the coming challenges in biology, and our speculative dreams for the future of what biology could look like in the years ahead.

🔊 Listen to “How new communities are propelling the future of tech + bio”

Then, we have Shaq on again in a short “Securities” episode looking at the “denominator effect” and how today’s global macro conditions are changing the VC asset class.

🔊 Listen to “VC 101: the denominator effect”

Lux Recommends

That’s it, folks. Have questions, comments, or ideas? This newsletter is sent from my email, so you can just click reply.

continue
reading