Image Credits: Bill Smith / Flickr / Creative Commons
How do we come to agree?
The big news over here at Lux this week was that Grace Isford joined us as our newest investor, based in New York City. She’s going to be investing at the intersection of web3/crypto, data infrastructure and AI/ML, an area where there is an incredible amount of deep tech being built such as the future of information theory and distributed consensus. Welcome Grace!
It’s on the latter subject of consensus that I want to explore more today, since it’s perhaps the single most important challenge facing democratic life today. Consensus, or the ability of different people to agree to the same set of facts and conclusions, is the key property of any functioning social system. When it works, decisions are rapidly made, morale is high, direction is clear and social organizations can be quite resilient and effective. A lack of consensus leads to stasis and sclerosis, of trying to “build” consensus rather than doing the actual work of execution.
In science, consensus is the acceptance of experimental results into the canon of a particular field. Peter Galison, the renowned historian of science at Harvard, built his career on a book titled “How Experiments End.” A scientist conducts an experiment and gets a result, perhaps a result that upends some accepted facts within a discipline. What happens next? A single experiment with a novel result could just be a fluke, a statistical oddity or perhaps a misconfigured instrument. So the scientist does another experiment, maybe with the exact same procedures or perhaps with a different approach that would make the result more resilient to criticism.
Such work begins the recursion at the heart of science: more evidence is collected, more scientists get involved. But when do we know that the experiment has been validated? And conversely, when should we throw out previously accepted results that now appear to be overturned? Galison gives a tremendously erudite depiction of this subject (one best left to reading the actual book), but the stupidly simple answer is: it really depends, and it’s complicated. Underlying that complexity though is the process of science itself, which handles divergent views in a specific way in order to synthesize a consensus.
We see this dynamic in VC partner meetings. There are procedures, guidelines and benchmarks to make a decision, systems that are designed to help a group of diverse and divergent investors come to consensus around an opportunity. Yet, every investment is unique, each in a different market with a different product sitting at a different stage. There’s an equivalent “How Investment Decisions End” — every time, the consensus function needs to be adapted to meet the needs of each company that presents and ensure the most accurate and highest conviction decisions. Divergence of opinion is a hallmark of any great investment firm.
Yet, some of the strongest mechanisms here don’t have any divergence at all. Distributed consensus is one of the most powerful developments that blockchain technology offers, and it’s part of the reason why web3 proponents can at times feel almost religious in their fervency. The ability of otherwise unrelated computers to come to an agreement around “state” (i.e. the data on the blockchain) without referencing the same, shared dataset is an incredibly powerful tool — the exact tool, in fact, that scaffolded the very civilization we are a part of.
Yet, even as computers build better and more robust consensus mechanisms, not just to compute token ownership but also to validate external information through oracles and other approaches, human society itself is tapering its abilities to generate consensus.
Take a look at Russian aggression against Ukraine the past few weeks. This week, markets gyrated as the media disseminated news that Russia was pulling back troops from the Ukrainian border on Tuesday, with the S&P 500 up on Tuesday and reaching a weekly high on Wednesday. Then U.S. intelligence agencies announced that Russia had in fact buttressed its forces on Ukraine’s border all along, sending shares tumbling about 4% over Thursday and Friday’s trading sessions.
Getting an accurate assessment on Russian troop levels is not an easy problem, but it’s also not something that only a handful of intelligence agencies can do. Planet Labs (a Lux family company), Capella Space, and other commercial satellite imagery providers have the means to capture and track Russia’s movements for the general public. Are there troop carriers moving? Where are they moving to? What’s their capacity?
All of us can verify more of that information than ever before. Take one example of a bridge analyzed by Nikkei Asia:
A military pontoon bridge appears over the Pripyat River, less than 6 km from Ukraine’s northern border with Belarus, in commercial satellite imagery taken Tuesday by U.S.-based Maxar Technologies. In images from U.S.-based satellite operator Planet Labs, the structure is absent on Monday but present Tuesday.
While disinformation and misinformation are both key elements of Russia’s strategy in this phase of the contest, it’s not as if consensus is impossible. There is now enough density of satellite sensing, on-the-ground video sources, and open-source intelligence that anyone can now develop an independent view on the conflict. There may well be a range of views given the available evidence, but there is more than enough data to make finding an “end” to the analysis possible.
What happens though when the information has been analyzed, the results are in, and yet consensus remains elusive? This has been the crisis of American politics. On a wide range of issues, debates are everlasting and ultimately fruitless, leading to irreconcilable differences as I highlighted a few weeks ago on "Securities” in “American Civil War 2.0.”
Why can we find scientific consensus, investment consensus, token consensus on the blockchain and even mostly consensus around Russian troop movements, but not around areas like health or immigration or infrastructure or climate change?
It’s simple: most of the consensus has already been established in the fields among the former. Scientists share a common set of principles, experimental approaches, and procedures to drive toward consensus. Venture funds like Lux, even if they are heterogeneous in making decisions, share a common perspective on what makes a distinguished investment and the kinds of evidence that should exist to identify them. Blockchain protocols rigorously apply consensus algorithms to their database, and analysts of Ukraine have constraints and norms that push them to at least some form of consensus.
Society, meanwhile, doesn’t have all those layers of consensus to build upon for new decisions. There’s no algorithmic blockchain ensuring that the basic facts of reality are cross-validated, or the scientific method to ensure that evidence is considered with appropriate context. Consensus is recursive, and without better consensus functions around values and tradeoffs, it’s impossible for a nation to make decisions.
America and much of the West escaped that lack of deep consensus simply through abundance. Multiple values and tradeoffs could be supported by building institutions that allowed them all to coexist. Venture funds can’t make every investment and scientists can’t accept every contradictory result, but wealthy nations can support a wide range of consensuses, each underwritten in parallel.
Despite the optimism that emanates from the business self-help bookshelf, building a singular consensus across a nation is monstrously hard work. Yet, if we are to lift above malaise and build a new future, we do actually have to agree on what that new future should look like and what paths are available for us to get there. Without more of that critical work, we might have a new book to write: “How society ends.”
Conjunction Junction, what’s your function?
Talking about consensus and dissensus, the theme of Lux’s quarterly LP letter for this past quarter was “the power of ‘and’.” The core message is that we are moving from a period of conjunction to one of disjunction — the crowded wisdom of everyone investing in the same basket of high-growth SaaS stocks is being supplanted by a varied strategy as investors in the public markets and even among venture firms scramble to find their own alpha in this much more complicated, post-Covid world.
Josh Wolfe posted excerpts of the LP quarterly letter to Twitter on Thursday — if you didn’t catch some of the highlights, definitely check them out.
The power of And: AI + fusion
The ITER tokamak device. Image Credits: Oak Ridge National Laboratory / Flickr / Creative Commons
Last week in "Securities,” I wrote about positive developments in nuclear energy in “Fission today, fusion tomorrow.” Nuclear fission got a boost from France, which announced it would build a series of new reactors over the next decade. Meanwhile, scientists in the United Kingdom sustained a record-breaking fusion reaction for about five seconds — proving that the design of the International Thermonuclear Experimental Reactor (ITER) in southern France is workable (of course, we'll know for sure when ITER launches, since that is how experiments end).
There was further bold news on the fusion front this week. DeepMind, which is owned by Alphabet, published a paper in Nature on Tuesday that demonstrated how deep reinforcement learning could be used to control the superheated plasma in a tokamak device — the exact type of doughnut-shaped magnetic containment vessel that is at the center of the ITER.
The plasma in a fusion reactor is too hot to be contained by any form of traditional material. So instead, a tokamak uses magnetic power to contain plasma within a very strict bound of a vacuum at the center of a doughnut-shaped metal shell. The magnets have to be carefully and continuously calibrated to contain the plasma though, which is unstable and can breach the vacuum at any time if not precisely contained.
DeepMind showed that its AI model could adjust the voltage of those magnets thousands of times per second in order to optimally contain the plasma. That should improve the viability for sustained fusion power — bringing the world one small inch closer to abundant energy and an entirely new energy security paradigm.
Lux Recommends
Shaq Vayda recommended this Wall Street Journal piece by Christopher Mims looking at the future of physical stocking technology and its effect on supply chains.
Shaq also recommended this new paper in Science, where scientists successfully used a technique known as ex vivo lung perfusion to convert a lung from blood type A to blood type O. By removing the blood antigens, the transformed lungs could be used by any recipient, expanding eligibility for organs, particularly patients with type O blood who have the least access.
Adam Goulburn recommends this documentary video by DeepMind that covers the company’s development of AlphaFold, which used deep reinforcement learning to solve the challenge of predicting how proteins will fold based on their underlying amino acids.
Sam Arbesman recommends this essay by Michael Nielsen and Kanjun Qiu that explores outliers in the scientific literature and how best to build a scientific funding system to encourage these great papers.
Every few years, there is a great critical teardown of TED Talks and their progeny. Oscar Schwartz joins this tradition with an essay in The Drift, dichotomizing the facile and gauzy optimism of the talks with the tough and deep innovation that the world actually needs.
That’s it, folks. Have questions, comments, or ideas? This newsletter is sent from my email, so you can just click reply.
Forcing China’s AI researchers to strive for chip efficiency will ultimately shave America’s lead
In incididunt ad qui nostrud sint ullamco. Irure sint deserunt Lorem id officia dolore non. Anim dolor minim sit dolor et sint aliquip qui est. Ex in tempor laborum laboris dolor laboris ullamco quis. Enim est cupidatat consequat est culpa consequat. Fugiat officia in ea ea laborum sunt Lorem. Anim laborum labore duis ipsum mollit nisi do exercitation. Magna in pariatur anim aute.
In incididunt ad qui nostrud sint ullamco. Irure sint deserunt Lorem id officia dolore non. Anim dolor minim sit dolor et sint aliquip qui est. Ex in tempor laborum laboris dolor laboris ullamco quis. Enim est cupidatat consequat est culpa consequat. Fugiat officia in ea ea laborum sunt Lorem. Anim laborum labore duis ipsum mollit nisi do exercitation. Magna in pariatur anim aute.
Right now, pathbreaking AI foundation models follow an inverse Moore’s law (sometimes quipped “Eroom’s Law”). Each new generation is becoming more and more expensive to train as researchers exponentially increase the number of parameters used and overall model complexity. Sam Altman of OpenAI said that the cost of training GPT-4 was over $100 million, and some AI computational specialists believe that the first $1 billion model is currently or will shortly be developed.
As semiconductor chips rise in complexity, costs come down because transistors are packed more densely on silicon, cutting the cost per transistor during fabrication as well as lowering operational costs for energy and heat dissipation. That miracle of performance is the inverse with AI today. To increase the complexity (and therefore hopefully quality) of an AI model, researchers have attempted to pack in more and more parameters, each one of which demands more computation both for training and for usage. A 1 million parameter model can be trained for a few bucks and run on a $15 Raspberry Pi Zero 2 W, but Google’s PaLM with 540 billion parameters requires full-scale data centers to operate and is estimated to have cost millions of dollars to train.
Admittedly, simply having more parameters isn’t a magic recipe for better AI end performance. One recalls Steve Jobs’s marketing of the so-called “Megahertz Myth” to attempt to persuade the public that headline megahertz numbers weren't the right way to judge the performance of a personal computer. Performance in most fields is a complicated problem to judge, and just adding more inputs doesn't necessarily translate into a better output.
And indeed, there is an efficiency curve underway in AI outside of the leading-edge foundation models from OpenAI and Google. Researchers over the past two years have discovered better training techniques (as well as recipes to bundle these techniques together), developed best practices for spending on reinforcement learning from human feedback (RLHF), and curated better training data to improve model quality even while shaving parameter counts. Far from surpassing $1 billion, training new models that are equally performant might well cost only tens or hundreds of thousands of dollars.
This AI performance envelope between dollars invested and quality of model trained is a huge area of debate for the trajectory of the field (and was the most important theme to emanate from our AI Summit). And it’s absolutely vital to understand, since where the efficiency story ends up will determine the sustained market structure of the AI industry.
If foundation models cost billions of dollars to train, all the value and leverage of AI will accrue and centralize to the big tech companies like Microsoft (through OpenAI), Google and others who have the means and teams to lavish. But if the performance envelope reaches a significantly better dollar-to-quality ratio in the future, that means the whole field opens up to startups and novel experiments, while the leverage of the big tech companies would be much reduced.
The U.S. right now is parallelizing both approaches toward AI. Big tech is hurling billions of dollars on the field, while startups are exploring and developing more efficient models given their relatively meagre resources and limited access to Nvidia’s flagship chip, the H100. Talent — on balance — is heading as it typically does to big tech. Why work on efficiency when a big tech behemoth has money to burn on theoretical ideas emanating from university AI labs?
Without access to the highest-performance chips, China is limited in the work it can do on the cutting-edge frontiers of AI development. Without more chips (and in the future, the next generations of GPUs), it won’t have the competitive compute power to push the AI field to its limits like American companies. That leaves China with the only other path available, which is to follow the parallel course for improving AI through efficiency.
For those looking to prevent the decline of American economic power, this is an alarming development. Model efficiency is what will ultimately allow foundation models to be preloaded onto our devices and open up the consumer market to cheap and rapid AI interactions. Whoever builds an advantage in model efficiency will open up a range of applications that remain impractical or too expensive for the most complex AI models.
Given U.S. export controls, China is now (by assumption, and yes, it’s a big assumption) putting its entire weight behind building the AI models it can, which are focused on efficiency. Which means that its resources are arrayed for building the platforms to capture end-user applications — the exact opposite goal of American policymakers. It’s a classic result: restricting access to technology forces engineers to be more creative in building their products, the exact intensified creativity that typically leads to the next great startup or scientific breakthrough.
If America was serious about slowing the growth of China’s still-nascent semiconductor market, it really should have taken a page from the Chinese industrial policy handbook and just dumped chips on the market, just as China has done for years from solar panel manufacturing to electronics. Cheaper chips, faster chips, chips so competitive that no domestic manufacturer — even under Beijing direction — could have effectively competed. Instead we are attempting to decouple from the second largest chips market in the world, turning a competitive field where America is the clear leader into a bountiful green field of opportunity for domestic national champions to usurp market share and profits.
There were of course other goals outside of economic growth for restricting China’s access to chips. America is deeply concerned about the country’s AI integration into its military, and it wants to slow the evolution of its autonomous weaponry and intelligence gathering. Export controls do that, but they are likely to come at an extremely exorbitant long-term cost: the loss of leadership in the most important technological development so far this decade. It’s not a trade off I would have built trade policy on.
The life and death of air conditioning
Photo by Ashkan Forouzani on Unsplash
Across six years of working at TechCrunch, no article triggered an avalanche of readership or inbox vitriol quite like Air conditioning is one of the greatest inventions of the 20th Century. It’s also killing the 21st. It was an interview with Eric Dean Wilson, the author of After Cooling, about the complex feedback loops between global climate disruption and the increasing need for air conditioning to sustain life on Earth. The article was read by millions and millions of people, and hundreds of people wrote in with hot air about the importance of their cold air.
Demand for air conditioners is surging in markets where both incomes and temperatures are rising, populous places like India, China, Indonesia and the Philippines. By one estimate, the world will add 1 billion ACs before the end of the decade. The market is projected to before 2040. That’s good for measures of public health and economic productivity; it’s unquestionably bad for the climate, and a global agreement to phase out the most harmful coolants could keep the appliances out of reach of many of the people who need them most.
This is a classic feedback loop, where the increasing temperatures of the planet, particularly in South Asia, lead to increased demand for climate resilience tools like air conditioning and climate-adapted housing, leading to further climate change ad infinitum.
Josh Wolfe speaking at Nvidia Auditorium at Stanford. Photo by Chris Gates.
Josh Wolfe gave a talk at Stanford this week as part of the school’s long-running Entrepreneurial Thought Leaders series, talking all things Lux, defense tech and scientific innovation. The .
Lux Recommends
As Henry Kissinger turns 100, Grace Isford recommends “Henry Kissinger explains how to avoid world war three.” “In his view, the fate of humanity depends on whether America and China can get along. He believes the rapid progress of AI, in particular, leaves them only five-to-ten years to find a way.”
Our scientist-in-residence Sam Arbesman recommends Blindsight by Peter Watts, a first contact, hard science fiction novel that made quite a splash when it was published back in 2006.
Mohammed bin Rashid Al Maktoum, and just how far he has been willing to go to keep his daughter tranquilized and imprisoned. “When the yacht was located, off the Goa coast, Sheikh Mohammed spoke with the Indian Prime Minister, Narendra Modi, and agreed to extradite a Dubai-based arms dealer in exchange for his daughter’s capture. The Indian government deployed boats, helicopters, and a team of armed commandos to storm Nostromo and carry Latifa away.”
Sam recommends Ada Palmer’s article for Microsoft’s AI Anthology, “We are an information revolution species.” “If we pour a precious new elixir into a leaky cup and it leaks, we need to fix the cup, not fear the elixir.”
I love complex international security stories, and few areas are as complex or wild as the international trade in exotic animals. Tad Friend, who generally covers Silicon Valley for The New Yorker, has a great story about an NGO focused on infiltrating and exposing the networks that allow the trade to continue in “Earth League International Hunts the Hunters.” "At times, rhino horn has been worth more than gold—so South African rhinos are often killed with Czech-made rifles sold by Portuguese arms dealers to poachers from Mozambique, who send the horns by courier to Qatar or Vietnam, or have them bundled with elephant ivory in Maputo or Mombasa or Lagos or Luanda and delivered to China via Malaysia or Hong Kong.”
