On Recursive Self-Improvement (Part I) - by Dean W. Ball

Frontier AI labs are already automating large portions of their research and engineering work, and within 1-2 years their effective workforces will balloon from thousands to hundreds of thousands - except most of those workers won't be human. These AI agents will have one job: make themselves smarter. OpenAI publicly plans for hundreds of thousands of automated research interns within nine months and a fully automated workforce in two years. This isn't aspirational - current models already write most of the code at frontier labs, and staff confirm this openly.

The key question isn't whether this happens, but what it means when it does. One scenario: AI capabilities improve faster but stay within the current paradigm - like a Bugatti going from 200 to 300 mph. Fast enough that people inside notice, but bystanders just see "extremely fast" twice. The public won't track the difference and will complain that AI automation didn't deliver on promises. The other scenario: fundamental phase change. The Bugatti learns to fly, reduces its price by 99%, and starts working on underwater operation - all on its own, with humans just operating the controls it built for them.

The uncertainty centers on diminishing returns versus untapped efficiency gains. Skeptics point to scaling laws - each order of magnitude more compute yields smaller improvements, from 9% to 90% to 99% to 99.9%. Bulls counter that algorithmic efficiency gains are still massive and underexplored. Human-driven research achieves roughly 400% efficiency improvements per year through architecture tweaks, dataset improvements, and infrastructure enhancements. Most AI research is grinding through these incremental gains, not pie-in-sky paradigm shifts. Labs are talent-constrained - they pay top personnel tens or hundreds of millions for a reason.

Imagine a lab with 1000 staff: 800 grinding on current paradigm improvements, 200 exploring new directions. Both jobs reduce to designing experiments, writing code, analyzing results. Models already do much of this work and improve constantly. They fall short on multi-day experiment reliability and generating brilliant research insights - but do they need to excel at the latter? A human researcher with an army of automated junior researchers testing variations could be transformative even if the models never have original insights. The real question is whether there's genuinely more low-hanging fruit to find (10x researchers means 4000% annual efficiency gains) or whether humans were already finding most practical gains (automation just accelerates discovery of the same improvements). Either way, policymakers need to pay attention - with minimal hysterics and maximum technocratic seriousness - because this changes everything about AI development dynamics.