We've Been Building Ghosts All Along?
Why AI Feels Both Amazing and Frustrating (And Will for Years) - Andrej has answers.
I just finished watching Andrej Karpathy’s latest conversation, and his one statement hit me like a freight train: we’re not building digital humans—we’re building ghosts.
This might sound like semantic wordplay, but the more you think on it, it’s actually the key to understanding why AI timelines keep surprising everyone (in both directions), why coding agents work but your PowerPoint bot doesn’t, and why the path to AGI looks nothing like the sci-fi script we’ve been reading from.
The Ghost in the Machine
Here’s Karpathy’s crucial insight: animals evolved through a very different optimization process than LLMs. When a zebra is born, it’s running within minutes—not because it learned to run, but because evolution encoded that capability directly into its neural architecture through millions of years of natural selection. That’s hardware, not software.
LLMs, by contrast, are trained by imitating the digital exhaust of human civilization—internet documents, code repositories, conversations. They’re not animals. They’re “ethereal spirit entities,” as Karpathy puts it, that exist purely in the realm of bits and patterns. They’re ghosts of human knowledge, not biological intelligences.
This distinction matters enormously. It means we can’t simply map animal learning onto AI development. It means the cognitive architecture of these systems is fundamentally different from ours. And it means we need to stop making lazy analogies to “kindergarteners” or “high schoolers” when describing model capabilities.
The March of Nines
The second revelation: every improvement in AI is a constant amount of work, regardless of where you are on the capability curve.
Karpathy calls this “the march of nines.” Getting from 90% to 99% accuracy takes just as much effort as getting from 99% to 99.9%. This is why self-driving took a decade despite perfect demos in 2014. This is why coding agents are impressive but not yet replacing senior engineers. This is why ChatGPT can write a sonnet but can’t reliably book you a vacation.
Each nine represents thousands of edge cases, corner cases, integration challenges, and silent failures that only emerge when the system meets reality. The demo-to-product gap isn’t a gap—it’s a canyon that requires constant, grinding work to cross.
Why Code? Why Now?
So why do coding agents work (relatively) well while agents for slides, customer service, or strategic planning still feel half-baked?
Because code has always been text.
The entire edifice of software engineering—IDEs, version control, diff tools, test suites, CI/CD pipelines—was already built around manipulating text. LLMs are exquisite text processors. The fit is natural.
But more importantly, code is structured, verifiable, and operates in a closed world. You can run tests. You can see if it compiles. The feedback loop is immediate and binary. Compare this to making a slide deck, where “success” is subjective, culturally dependent, and requires sophisticated aesthetic judgment.
This is why API revenue is dominated by coding applications. Not because LLMs are narrow tools, but because coding was accidentally pre-adapted for the specific cognitive architecture of transformer-based language models.
The Slop Problem (Or: Why Your AI Tutor Isn’t Here Yet)
Karpathy wants to build an AI tutor that matches his experience learning Korean from a skilled human teacher. Here’s what that teacher did:
Instantly diagnosed exactly what he knew and didn’t know
Served him material at the precise edge of his capability
Never gave him something too hard (frustrating) or too easy (boring)
Made him feel like he was the only constraint on his learning
Current LLMs can’t do this. Not even close. They’re “collapsed”—they sample from a narrow manifold of responses. Ask ChatGPT for a joke and you’ll get the same three jokes. Ask for reflection on a book chapter and you’ll get variations on the same shallow take.
This collapse isn’t just annoying—it’s fundamental. LLMs are too good at memorization and too eager to regurgitate. They produce what Karpathy calls “slop”: plausible-sounding content that lacks genuine insight or novelty. They can’t maintain the entropy needed for true exploration of ideas.
The technical challenge isn’t just making them smarter. It’s making them more diverse without letting them drift into incoherence. It’s teaching them to know what they don’t know. It’s giving them something analogous to intellectual humility.
What This Means for Timelines
If you’re building an AI company or investing in one, here’s what this framework implies:
The tasks that will be automated first:
Involve pure information processing (no physical component)
Have clear success criteria (can be verified programmatically)
Live in closed, well-defined domains
Benefit from the existing text-based tooling infrastructure
Can tolerate 80% automation with 20% human supervision
The tasks that will take another decade:
Require genuine creativity or taste
Span multiple domains with messy handoffs
Need “common sense” in truly novel situations
Have high costs of failure (safety-critical, security-critical)
Demand the kind of contextual judgment that comes from deep experience
The corporate knowledge worker doing PowerPoint and email isn’t getting replaced next year. The consultant synthesizing disparate information into strategic recommendations isn’t either. But the junior developer writing CRUD applications? The call center employee following scripts? The paralegal doing document review? Those roles are in the crosshairs right now.
The Intelligence We’re Missing
What’s still missing from these systems? Karpathy has a list:
Continual learning: The ability to update themselves based on experience without full retraining
Genuine reflection: Going beyond surface-level pattern matching to meta-cognitive reasoning
Cultural accumulation: Building a shared knowledge base that compounds over generations of models
Self-play and competition: The evolutionary pressure that drove biological intelligence
Emotional and motivational systems: The instincts and drives that guide exploration vs. exploitation
We have cortical tissue (the transformer). We have something like a prefrontal cortex (chain-of-thought reasoning). But we’re missing the basal ganglia, the hippocampus, the amygdala—all the subcortical structures that make human intelligence human.
The Real Timeline
So when does this all come together? Karpathy’s answer is refreshingly honest: probably not as fast as VCs need, but faster than pessimists expect.
He’s not betting on a discrete jump to AGI. He’s betting on the same hyper-exponential curve humanity has been riding since the Industrial Revolution. AI isn’t a discontinuity—it’s an acceleration. Just like computers, just like the internet, just like mobile phones.
The pattern is always the same: miraculous demos, then years of grinding through the march of nines, then gradual diffusion throughout the economy, then complete transformation of how we work and live. We’re in the middle of that cycle right now, mistaking demos for products and hype for progress.
But make no mistake: the transformation is real. We’re just on a longer fuse than most people think.
Building in the Gap
If you’re building AI products today, the opportunity isn’t in replacing humans wholesale. It’s in creating the interfaces and workflows for human-AI collaboration.
It’s building the “operation centers” that sit behind the autonomous systems—the human oversight layer that handles the 1% of cases the AI can’t. It’s designing the autonomy slider that lets companies dial up AI assistance as the technology matures. It’s creating the feedback loops that turn deployment experience back into training data.
The winners in this space won’t be the ones with the best AI. They’ll be the ones with the best understanding of where the march of nines currently sits, and what humans and machines each do best at this particular moment in time.
The Education Bet
Karpathy’s building Eureka Labs to create what he calls “Starfleet Academy”—an elite institution for technical learning in the age of AI. His thesis: even in a post-AGI world, humans will want to learn for the same reason they go to the gym today.
Not because we need physical strength (we have machines for that), but because it’s fun. Because it’s how we signal status. Because it’s how we connect with something primal in our nature.
If he’s right, the highest value education won’t be vocational training for jobs that AI will automate anyway. It’ll be the pursuit of human excellence for its own sake—the cognitive equivalent of running a marathon or climbing a mountain.
The market for this might be smaller than mass education. But it might also be the only education that matters.
The Bottom Line
We’re not in the year of agents. We’re in the decade of agents. The march of nines continues. The ghosts are getting smarter, but they’re still ghosts.
Build accordingly.
Here’s the full podcast: