Electron microscopic image of the 1976 isolate of Ebola virus. Photo by CDC on Unsplash
As bio risks rise, the Pentagon publishes its first-ever Biodefense Posture Review
Few global systemic risks get the blood as flowing metaphorically and literally as bioweapons and pandemics. It’s the intersectional nightmare of popular culture (Planet of the Apes, Contagion, Train to Busan, I Am Legend) and Langley’s most astute prognosticators. Part of their allure for filmmakers (and the intensity of the fear for security analysts) is their inherent craftiness. A nuclear weapon, despite immense radiological and fire damage, is ultimately a really, really big bomb, but a virus — that’s omnipresent, impossible to see with the human eye, and dastardly hard to stop once it gets going.
Covid-19 centered the world’s fleeting attention on systemic pandemic risk, but it’s a risk that unfortunately continues to expand in scope and danger.
Why? First and foremost, multiple countries — most notably North Korea, China, Russia and Iran — continue to actively pursue offensive bioweapons programs or are believed to have such programs available for activation. Biological weapons were banned with the signing of the Biological Weapons Convention in 1972, but adherence to its protocols is most definitely not uniform around the world.
Access to samples of deadly pathogens as well as sophisticated laboratory tools continues to expand around the world as more and more countries develop and devote resources to health and the biological sciences. That makes it easier for mistakes (“lab leaks”) to occur even with the most robust safeguards, and also expands the number of targets for the intentional theft of these potentially potent weapons.
Perhaps most ominously, climate change is increasing biological and pandemic risks in a multitude of ways. Greater heat can amplify the spread of pathogens, human migration is increasingly expanding the scope of local and regional outbreaks, and human population growth into formerly wild areas of the world are creating new interfaces between undiscovered diseases and humans. In the most imaginative frontiers, the melting of permafrost is unearthing fossils and remnants of the past, which some analysts fear could release and spread ancient diseases that human immune systems have long since forgotten how to fight.
Unlike most threats that America’s armed forces confront, pandemics are undirected, exponentially growing, and require vast coordination across bureaucratic and national borders as well as civilian and military leaders to combat. There is no concept of “front lines” and “behind the lines” — every person can potentially be exposed and become a vector for transmission, which adds profound complexity to defense planning.
The Pentagon has struggled to shore up its approach to these growing risks even after it accelerated action in the wake of Covid-19. To that end, last week the Department of Defense released its first-ever Biodefense Posture Review, an initiative that is designed to evaluate how the objectives published in the White House’s National Biodefense Strategy from October 2022 connects with the department’s own performance and future planning.
The Pentagon describes it’s chief challenge: “The revealed that, although DoD possesses the necessary authorities for biodefense, it could benefit from a more collective and unified approach to coordinating its biodefense roles and responsibilities due to the decentralized nature of the biodefense enterprise…” Later, the review notes, “DoD is not sufficiently engaging with interagency partners and allies and partners to build global biodefense capabilities, enhance posture, maximize interoperability, and strengthen campaigning.”
Not exactly revelatory, but the review does set the parameters for some of the risk complexities that come out of bio.
One of those is simply the element of surprise. Pandemics can arrive from anywhere and can act vastly different from one outbreak to the next. As the review notes, “The future threat landscape requires moving beyond the historical ‘threat list’ approach for capability development to more effectively and rapidly respond to biothreats (naturally occurring, accidental, or deliberate).” In other words, the Pentagon can’t just be looking for signs of Ebola or Marburg, but instead must have pandemic surveillance systems designed to catch subtle changes in a population’s health that might indicate the arrival of a known disease — or something new entirely.
There are more complexities though, even if better information processing is implemented. The review notes that more work needs to be done to train staff on how to deal with biothreats (whether natural or man-made) that are coupled with foreign disinformation campaigns. Covid-19 highlighted both the polarization around public health in the United States and elsewhere, as well as the ability of overseas influence operations to sow general discord. The review emphasizes the need to develop “best practices to separate the truth from fiction” (probably a good bumper sticker for most of government).
A less surprising observation is that America has a deep dependency on China for critical bio and health products. What’s more surprising though is that the Pentagon seems to be increasingly understanding how challenging the business environment is to fulfill the Department of Defense’s needs:
For example, the bulk of production, especially for key precursor materials, has moved overseas (especially to China). Subsequently, in many cases, domestic production has dwindled to a single supplier. Resultantly, investors increasingly find investing in the domestic biodefense sector unattractive, which further erodes the infrastructure needed to support DoD requirements. Additionally, the production workforce has shifted, leaving a dearth of talent in the United States. Compared to the global market, DoD’s unique biodefense demands are small and not commercially competitive.
That’s heartening to hear, because far too often, the Pentagon and its backers can sound almost hysterical about the lack of business support for the defense mission. Businesses rely on sound economics — and large swaths of Pentagon procurement are unsound, particularly in emerging technologies.
Finally, one nice intersection with the work we are doing with the Lux Riskgaming Initiative was highlighted by the review, which “acknowledged the value of table-top exercises as key contributors to integrated risk assessments across the biodefense space.” Given the complexity of the subject, bringing a wide variety of stakeholders around the table to comprehend and take action on difficult scenarios is an invaluable medium for building capacity and talent.
It’s excellent news that the Defense Department is building up its capabilities against viruses and other pathogens. Yet, so much more needs to be done, both internally and across the whole of government. Particularly in the United States, where public health agencies tend to concentrate at the county level of government, that coordination problem remains profoundly challenging. It shouldn’t take 2-3 pandemics to get us ready for the next one; instead, let’s try to build the systems we know we need to prevent them before they spread.
Video: The Terrifying Fusion: AI, Bioweapons, and the Future of Warfare
Design by Chris Gates
In fortuitous timing, the Pentagon’s Biodefense Posture Review was published at almost the exact same time as our most recent “Securities” video, which is focused on the dual-use challenges of AI/ML tools in biochemistry. These new products offer incredible opportunities to advance human health, but also potentially add new pathways for bad actors to build more lethal and more hidden weapons programs.
It’s easy to look at this risk and just assume everything leads to dystopia, but the reality is quite a bit more complicated. Given that there are already extremely virulent bioweapons and lethal chemical weapons available, what more can AI do? And what other capabilities does AI offer that could transform the threat environment? Watch the video and learn more, and also check out last year’s “Securities” issue, “AI, dual-use medicine, and bioweapons.”
Generative AI has enhanced cybersecurity weapons, but defenders can still claim a lot of value
Our summer associate Yadin Arnon combed through the cybersecurity industry, where investor focus has intensified amidst the rapid rise of generative AI. Yadin wrote up his analysis on the market for us, before heading back to the University of Chicago Booth School of Business for the second year of his MBA.
For many adversaries, especially the financially-motivated ones, hacking is a numbers game. With tools that dramatically speed up malicious campaigns, the time to exploit a target is narrowing. So what can we do?
Fortunately, this revolution in AI leaves plenty of room for security innovation across the entire attack surface, with different types of assistants and platforms offering intelligent alerting and attack prevention. I believe that the following five areas in cyber have the largest opportunity to produce cutting-edge novelty in the age of growing threats leveraging machine learning and generative AI: 1) workforce security, 2) AI model security, 3) identity and access management, 4) software supply chain security and 5) security operations.
Lux Recommends
Congratulations to the scientists and engineers behind Chandrayaan-3, India’s spacecraft which successfully landed on the moon’s south pole. India is the first nation to reach this region — which is theorized to hold frozen water — and is only the fourth nation to successfully land a spacecraft on the moon (the others are the United States, Russia and China).
Tess Van Stekelenburg highlights a new Nature review article by a group of researchers who explore "Scientific discovery in the age of artificial intelligence.” “Using Al for scientific innovation and discovery presents unique challenges compared with other areas of human endeavour where AI is utilized. One of the biggest challenges is the vastness of hypothesis spaces in scientific problems, making systematic exploration infeasible. For instance, in biochemistry, an estimated 1060 drug-like molecules exist to explore.”
Grace Isford points to SemiAnalysis editor Dylan Patel’s analysis of Nvidia (which had a blockbuster quarter of earnings for those who missed it). In Dylan’s post, "Nvidia’s Ramp – Volume, ASP, Cloud Pricing, Margins, EPS, Cashflow, China, Competition,” he highlights the performance difference between Nvidia and competitors. “For example, Super Micro disappointed lofty expectations after a >300% share price run-up this year with revenue guidance for the next quarter being flat. Their ramp in AI revenue isn’t being shown by them or many other AI head-fakes and it is violating the AI ramp narrative.”
Finally, MIT professor Yasheng Huang writes a fascinating argument in “How to Kill Chinese Dynamism” on the importance of liberal Hong Kong for mainland China’s economic development. With the recent political clampdown on the city’s freedoms, China’s growth has now stalled. “Those who believe that Chinese entrepreneurship somehow thrived under a magical formula of statism thus ignore the role that Hong Kong – and a number of other overseas domiciles – played in providing the conventional pillars of innovation-driven economic growth.”
That’s it, folks. Have questions, comments, or ideas? This newsletter is sent from my email, so you can just click reply.
Forcing China’s AI researchers to strive for chip efficiency will ultimately shave America’s lead
In incididunt ad qui nostrud sint ullamco. Irure sint deserunt Lorem id officia dolore non. Anim dolor minim sit dolor et sint aliquip qui est. Ex in tempor laborum laboris dolor laboris ullamco quis. Enim est cupidatat consequat est culpa consequat. Fugiat officia in ea ea laborum sunt Lorem. Anim laborum labore duis ipsum mollit nisi do exercitation. Magna in pariatur anim aute.
In incididunt ad qui nostrud sint ullamco. Irure sint deserunt Lorem id officia dolore non. Anim dolor minim sit dolor et sint aliquip qui est. Ex in tempor laborum laboris dolor laboris ullamco quis. Enim est cupidatat consequat est culpa consequat. Fugiat officia in ea ea laborum sunt Lorem. Anim laborum labore duis ipsum mollit nisi do exercitation. Magna in pariatur anim aute.
Right now, pathbreaking AI foundation models follow an inverse Moore’s law (sometimes quipped “Eroom’s Law”). Each new generation is becoming more and more expensive to train as researchers exponentially increase the number of parameters used and overall model complexity. Sam Altman of OpenAI said that the cost of training GPT-4 was over $100 million, and some AI computational specialists believe that the first $1 billion model is currently or will shortly be developed.
As semiconductor chips rise in complexity, costs come down because transistors are packed more densely on silicon, cutting the cost per transistor during fabrication as well as lowering operational costs for energy and heat dissipation. That miracle of performance is the inverse with AI today. To increase the complexity (and therefore hopefully quality) of an AI model, researchers have attempted to pack in more and more parameters, each one of which demands more computation both for training and for usage. A 1 million parameter model can be trained for a few bucks and run on a $15 Raspberry Pi Zero 2 W, but Google’s PaLM with 540 billion parameters requires full-scale data centers to operate and is estimated to have cost millions of dollars to train.
Admittedly, simply having more parameters isn’t a magic recipe for better AI end performance. One recalls Steve Jobs’s marketing of the so-called “Megahertz Myth” to attempt to persuade the public that headline megahertz numbers weren't the right way to judge the performance of a personal computer. Performance in most fields is a complicated problem to judge, and just adding more inputs doesn't necessarily translate into a better output.
And indeed, there is an efficiency curve underway in AI outside of the leading-edge foundation models from OpenAI and Google. Researchers over the past two years have discovered better training techniques (as well as recipes to bundle these techniques together), developed best practices for spending on reinforcement learning from human feedback (RLHF), and curated better training data to improve model quality even while shaving parameter counts. Far from surpassing $1 billion, training new models that are equally performant might well cost only tens or hundreds of thousands of dollars.
This AI performance envelope between dollars invested and quality of model trained is a huge area of debate for the trajectory of the field (and was the most important theme to emanate from our AI Summit). And it’s absolutely vital to understand, since where the efficiency story ends up will determine the sustained market structure of the AI industry.
If foundation models cost billions of dollars to train, all the value and leverage of AI will accrue and centralize to the big tech companies like Microsoft (through OpenAI), Google and others who have the means and teams to lavish. But if the performance envelope reaches a significantly better dollar-to-quality ratio in the future, that means the whole field opens up to startups and novel experiments, while the leverage of the big tech companies would be much reduced.
The U.S. right now is parallelizing both approaches toward AI. Big tech is hurling billions of dollars on the field, while startups are exploring and developing more efficient models given their relatively meagre resources and limited access to Nvidia’s flagship chip, the H100. Talent — on balance — is heading as it typically does to big tech. Why work on efficiency when a big tech behemoth has money to burn on theoretical ideas emanating from university AI labs?
Without access to the highest-performance chips, China is limited in the work it can do on the cutting-edge frontiers of AI development. Without more chips (and in the future, the next generations of GPUs), it won’t have the competitive compute power to push the AI field to its limits like American companies. That leaves China with the only other path available, which is to follow the parallel course for improving AI through efficiency.
For those looking to prevent the decline of American economic power, this is an alarming development. Model efficiency is what will ultimately allow foundation models to be preloaded onto our devices and open up the consumer market to cheap and rapid AI interactions. Whoever builds an advantage in model efficiency will open up a range of applications that remain impractical or too expensive for the most complex AI models.
Given U.S. export controls, China is now (by assumption, and yes, it’s a big assumption) putting its entire weight behind building the AI models it can, which are focused on efficiency. Which means that its resources are arrayed for building the platforms to capture end-user applications — the exact opposite goal of American policymakers. It’s a classic result: restricting access to technology forces engineers to be more creative in building their products, the exact intensified creativity that typically leads to the next great startup or scientific breakthrough.
If America was serious about slowing the growth of China’s still-nascent semiconductor market, it really should have taken a page from the Chinese industrial policy handbook and just dumped chips on the market, just as China has done for years from solar panel manufacturing to electronics. Cheaper chips, faster chips, chips so competitive that no domestic manufacturer — even under Beijing direction — could have effectively competed. Instead we are attempting to decouple from the second largest chips market in the world, turning a competitive field where America is the clear leader into a bountiful green field of opportunity for domestic national champions to usurp market share and profits.
There were of course other goals outside of economic growth for restricting China’s access to chips. America is deeply concerned about the country’s AI integration into its military, and it wants to slow the evolution of its autonomous weaponry and intelligence gathering. Export controls do that, but they are likely to come at an extremely exorbitant long-term cost: the loss of leadership in the most important technological development so far this decade. It’s not a trade off I would have built trade policy on.
The life and death of air conditioning
Photo by Ashkan Forouzani on Unsplash
Across six years of working at TechCrunch, no article triggered an avalanche of readership or inbox vitriol quite like Air conditioning is one of the greatest inventions of the 20th Century. It’s also killing the 21st. It was an interview with Eric Dean Wilson, the author of After Cooling, about the complex feedback loops between global climate disruption and the increasing need for air conditioning to sustain life on Earth. The article was read by millions and millions of people, and hundreds of people wrote in with hot air about the importance of their cold air.
Demand for air conditioners is surging in markets where both incomes and temperatures are rising, populous places like India, China, Indonesia and the Philippines. By one estimate, the world will add 1 billion ACs before the end of the decade. The market is projected to before 2040. That’s good for measures of public health and economic productivity; it’s unquestionably bad for the climate, and a global agreement to phase out the most harmful coolants could keep the appliances out of reach of many of the people who need them most.
This is a classic feedback loop, where the increasing temperatures of the planet, particularly in South Asia, lead to increased demand for climate resilience tools like air conditioning and climate-adapted housing, leading to further climate change ad infinitum.
Josh Wolfe speaking at Nvidia Auditorium at Stanford. Photo by Chris Gates.
Josh Wolfe gave a talk at Stanford this week as part of the school’s long-running Entrepreneurial Thought Leaders series, talking all things Lux, defense tech and scientific innovation. The .
Lux Recommends
As Henry Kissinger turns 100, Grace Isford recommends “Henry Kissinger explains how to avoid world war three.” “In his view, the fate of humanity depends on whether America and China can get along. He believes the rapid progress of AI, in particular, leaves them only five-to-ten years to find a way.”
Our scientist-in-residence Sam Arbesman recommends Blindsight by Peter Watts, a first contact, hard science fiction novel that made quite a splash when it was published back in 2006.
Mohammed bin Rashid Al Maktoum, and just how far he has been willing to go to keep his daughter tranquilized and imprisoned. “When the yacht was located, off the Goa coast, Sheikh Mohammed spoke with the Indian Prime Minister, Narendra Modi, and agreed to extradite a Dubai-based arms dealer in exchange for his daughter’s capture. The Indian government deployed boats, helicopters, and a team of armed commandos to storm Nostromo and carry Latifa away.”
Sam recommends Ada Palmer’s article for Microsoft’s AI Anthology, “We are an information revolution species.” “If we pour a precious new elixir into a leaky cup and it leaks, we need to fix the cup, not fear the elixir.”
I love complex international security stories, and few areas are as complex or wild as the international trade in exotic animals. Tad Friend, who generally covers Silicon Valley for The New Yorker, has a great story about an NGO focused on infiltrating and exposing the networks that allow the trade to continue in “Earth League International Hunts the Hunters.” "At times, rhino horn has been worth more than gold—so South African rhinos are often killed with Czech-made rifles sold by Portuguese arms dealers to poachers from Mozambique, who send the horns by courier to Qatar or Vietnam, or have them bundled with elephant ivory in Maputo or Mombasa or Lagos or Luanda and delivered to China via Malaysia or Hong Kong.”
