In 1965 I. J. Good predicted an intelligence explosion. In 2008 Eliezer Yudkowsky and Robin Hanson argued over whether it would come as a FOOM – one machine outgrowing the world in weeks – or as faster growth spread across a whole economy. In 2026 AI is being used to build AI, and the question is no longer whether the loop exists but where it runs. This essay argues that recursion has so far arrived closer to Hanson’s picture than to Yudkowsky’s, that this calls for vigilance rather than complacency, and that the right response is to make the loop visible, not to license it.
What the debate was really about
The FOOM debate is remembered as an argument about speed. Read again, it is at least as much about concentration. Both men agreed that machine intelligence would be a development of almost unprecedented impact and risk. What divided them was whether a single project could pull far enough ahead to dominate everyone else. When they met at Jane Street Capital in June 2011, the motion was exactly that: whether the first movers of an intelligence explosion would dominate more of the future than the first movers of farming and industry had.
Yudkowsky named the condition on which his own scenario depended. In a post of November 26, 2008, he argued that Douglas Engelbart’s computer tools had spread through the economy as a slow cascade rather than a concentrated takeoff. A concentrated FOOM, he explained, must absorb as much as possible of the processes that drive it. A project that cannot beat the world, he wrote, “may as well join it”. That sentence is the hinge of the whole question. FOOM is not only a claim about software. It is a claim that the entire production chain of intelligence can be captured in one place.
The strongest case for fear
The case for taking FOOM seriously deserves its full weight, because it is the strongest argument against the liberal approach to AI. Liability, markets and courts all work after the fact. They correct mistakes by letting harms happen, become visible and be priced. If a system could climb from roughly human level to overwhelming capability in weeks, there would be no after. Nobody can sue a takeoff. This is the coherent core of every proposal for prior permission – licenses for large training runs, compute thresholds, pauses: if the gap between warning and catastrophe may be shorter than any institution’s reaction time, rules have to act before it opens.
Paul Christiano’s 2018 essay on takeoff speeds sharpens this rather than softening it. A slow takeoff in his sense still compresses enormous change into a few years. It gives warning, but warning helps only a society that is watching and able to act. A liberal who dismisses FOOM should admit what the position rests on: an empirical bet that there will be time to learn.
What the evidence shows in 2026
The International AI Safety Report 2026 gives the best current view of that bet. It records that AI tools are now routinely used in AI development to write training code, design hardware and generate synthetic data. AI agents beat human experts on two-hour research-engineering tasks but do worse on eight-hour ones. A pilot study it cites, run by METR with the Forecasting Research Institute, asked whether any two-year period before 2029 would bring as much growth in effective compute as the six years from 2018 to 2024. Eight AI forecasting experts gave a median of 20 percent, ten superforecasters 8 percent. If AI were shown to beat human researchers on month-long projects, the superforecasters’ median rose to 18 percent.
These are not small numbers, and they should not be waved away. But they describe something other than FOOM. A roughly threefold acceleration of an already fast trend is far from the scenario Hanson attributed to Yudkowsky in 2008 – some twenty orders of magnitude of growth in about a week, a figure Yudkowsky himself said overstated the early phase. More important is where the loop runs. The report puts frontier training runs at about $500 million in compute, with the next generation expected to cost $1–10 billion. The inputs – chips, power, data centers, researchers, capital – are spread across a global supply chain and several competing companies. Each turn of the loop passes through procurement, hiring and engineering teams. That is recursion, but it is not a lone machine outgrowing the world.
Recursion as a social fact
The AIciety definition names recursive self-reinforcement as one of its four thresholds, and Level 9 of the AIciety stages is called recursion. Seen from there, FOOM is the engineering version of Level 9: recursion compressed into one machine and one week. What is happening instead is the sociological version. AI outputs become inputs to the processes that produce AI – code, chip designs, training data, evaluations – and, beyond the labs, to the institutions that fund and govern it. The loop closes through organizations before it closes inside a single model.
That has a practical consequence: a loop that runs through institutions can be observed. Rules should therefore attach to what can be seen and to what goes wrong – disclosure of how far a developer’s own research is automated, published measurements of AI research capability of the kind the report relies on, incident reporting, and liability for harm. None of this requires prior permission to train or to publish. It requires that the acceleration, if it comes, comes in public.
The same logic favors competition and open models. Hanson’s skepticism rested on the observation that progress usually comes from many contributions spread across an economy. The structural safeguard against a single winner is that no single actor holds all the inputs; many labs and open weights keep many loops running instead of one. The objection is serious: open models also spread capabilities to people who would misuse them, and a race can shorten the time available for safety work. But the alternative – a few closed actors running private, unobserved research loops – concentrates exactly the lead that the FOOM scenario fears.
What would change the argument
A position that rests on an empirical bet should say in advance what would make it lose. Three signals would count. First, evidence that AI systems outperform human researchers on month-long projects – the condition under which skeptical forecasters more than doubled their estimates. Second, a widening lead by one developer that rivals cannot close by buying the same inputs. Third, research automation that is used internally but no longer reported. Any of these would move the world toward Yudkowsky’s picture, and the case for earlier and stronger rules would grow with it. Updating on evidence is not a concession. It is what the liberal position claims to do better than its rivals.
What the song does with it
I’m Upping My P(doom) puts FOOM in its first chorus, alongside p(doom) and the Chinese Room. The lyricist osmarks glosses it as a runaway intelligence explosion and links the Hanson–Yudkowsky debate, adding that this is probably not where the word began. The idea returns in the final chorus as recursive self-improvement, and the annotations record that this line was contributed by Claude. A language model supplying the lyric about machines improving themselves is a small, harmless case of what this essay describes: AI output feeding back into the culture that argues about AI, with a human editor still in the loop. The widely shared video by mexicat, posted on YouTube on September 27, 2026, carried that loop far beyond the blogs where the debate took place.
Sources: I. J. Good, “Speculations Concerning the First Ultraintelligent Machine” (1965) · MIRI: The Hanson-Yudkowsky AI-Foom Debate (2013) · Paul Christiano: Takeoff speeds · International AI Safety Report 2026 · METR: Forecasting the Impacts of AI R&D Acceleration · osmarks: p(doom) song, objectively correct interpretation · Lexicon: FOOM
Leave a Reply