The AGI Illusion: Why We're As Far From Human Intelligence As Ancient Greeks Were From Boeing 747s

The Paradox of Optimism

Some of the smartest people of our time believe AGI is right around the corner. Even the more pessimistic ones, like Yann LeCun, believe it's attainable within our lifetimes. This optimism stands in stark contrast to our track record on other hard problems:

• Room Temperature Superconductors: Still theoretical after decades of research
• P vs NP: A fundamental computer science question that remains unsolved
• Quantum Gravity: No unified theory despite a century of effort

So why the optimism for AGI? At first, I thought it was simply money incentives. For a lot of people (Sam Altman and co.), it is. But that doesn't explain why researchers who don't stand to gain as much monetarily are equally optimistic.

The Case for Optimism: Emergent Capabilities and Existence Proof

The answer lies in two compelling observations.

First, emergent capabilities. GPT-2 barely had the ability to reason at all, and GPT-4 can pass the bar exam. All we did was throw more compute at it. Pretty amazing, really.

Second, existence proof. We already know intelligence exists. We know it runs on ~20 watts and evolved through a blind process. Evolution didn't need a research lab or a theory. Just time and trial and error.

By contrast:

• We can't solve P vs NP by building bigger computers
• Room temperature superconductors don't emerge from trying 10,000 more materials
• A theory of Quantum Gravity won't spontaneously appear even if we built the world's largest particle accelerator

These problems are fundamentally blocked by insight, not resources. AGI seems different. It appears to be a resource problem that we're actively throwing resources at.

So, Should We Be Excited or Pessimistic?

Should we embrace the hype? Or should we be extremely pessimistic, using dramatic oversimplifications like "BuT LLMs ArE JuSt ToKen PrEdIcToRs"?

Both, actually. Here's why.

The 747 Analogy: Existence ≠ Engineering

The reality is, we are as close to creating human intelligence as ancient Greeks were to building a Boeing 747. Birds existed, so they knew flight was possible. However, the gulf between "it exists in nature" and "we can engineer it" is often enormous. In the case of human intelligence, it's by definition the biggest gap we've ever attempted to cross.

The challenges are fundamental:

1. Current architectures can't do certain types of reasoning (particularly causal reasoning)
2. We need algorithmic breakthroughs we can't predict (by definition, if we could predict them, we'd have them)
3. Training paradigms may break down at larger scales (diminishing returns are already appearing)
4. The "easy" 80% of intelligence scales, but the last 20% requires entirely new approaches
5. We lack a real theory of what makes intelligence work

We're pouring 99% of resources into disembodied language models and calling it "the path to AGI." It's like trying to understand flight by making better and better propellers while ignoring aerodynamics.

What LLMs Have Actually Taught Us

What have we actually learned from LLMs? We've discovered their limitations. We can now extrapolate from observed constraints: there's no dataset, no architecture modification, no scale at which statistical learning over text (or text + other modalities) produces genuine causal reasoning.

But here's the critical insight: while we don't know all possible methods that can lead to causal reasoning, we do know for certain that embodied intelligence eventually gets there.

This suggests a different path forward.

The 302-Neuron Problem: Why Even a Worm Defeats Us

Let's examine the humble C. elegans, a tiny worm with exactly 302 neurons. We've completely mapped its connectome, yet we still can't fully simulate its behavior.

What We're Missing

Having the connectome is like having a circuit diagram without knowing what each component does:

• We don't know the strength of each connection (synaptic weights)
• We don't know which connections are excitatory vs inhibitory for many synapses
• We don't know the neuromodulator dynamics (chemical signals that change how neurons respond)
• We don't know the ion channel distributions that determine firing patterns
• Neurons aren't simple on/off switches. They have complex internal dynamics we don't fully understand

The Whack-A-Mole Problem

When OpenWorm researchers adjust parameters to make the simulated worm's locomotion more realistic, other behaviors degrade:

• Tune the muscle model so forward crawling looks right → the worm can't turn properly anymore
• Adjust neural timing to fix turning → now response to touch stimuli is wrong
• Fix sensory responses → locomotion rhythm breaks down

Why This Happens

The Deeper Implication: Intelligence May Not Be Compositional

This suggests that intelligence might not be compositional in the way we assume. We think: "solve vision + solve language + solve reasoning = AGI." But if these systems are deeply intertwined like the worm's behaviors, you can't just add them together.

This would mean:

• You can't benchmark progress on individual capabilities and extrapolate
• Scaling up subsystems doesn't guarantee they'll integrate properly
• The whole might require fundamentally different organization than the sum of parts

This is very different from engineering systems we understand:

• A bridge is compositional: design the supports, design the deck, put them together
• A computer is compositional: CPU, memory, storage work independently
• But a worm (and maybe intelligence) is holistic: everything depends on everything else in ways we can't disentangle

If a 302-neuron worm is holistic in this way, imagine an 86-billion-neuron human brain. The parameter space and interdependencies become incomprehensibly vast.

The Solution: Stop Throwing Darts, Recreate Initial Conditions

Here's the good news: we don't have to throw darts at a problem that likely isn't compositional in nature. All we have to do is recreate the initial conditions.

We need to build virtual worlds that perfectly mirror real ones. But not for humans.

We need to perfectly simulate the world that a 302-neuron worm experiences. This is called an umwelt.

The Umwelt Approach: Simulate the Sensory World, Not Physics

Here's the key insight: a C. elegans doesn't need Newtonian mechanics or fluid dynamics equations. It needs:

• Chemical gradients (food, danger, pheromones)
• Temperature differentials
• Mechanical pressure on its body
• Proprioceptive feedback from its own movement
• Maybe some vibration sensing

That's a much more tractable problem than "simulate reality." You're building the sensory manifold that 302 neurons evolved to process, not the universe that generates it.

The Test

Can you drop OpenWorm (or whatever simulated nervous system) into this virtual umwelt and get worm-like behavior? Not worm-like outputs to scripted inputs, but coherent adaptive behavior in a continuous environment where the body-brain-world loop closes.

This sidesteps the whole "is the neural simulation accurate" question. If the simulated worm navigates the simulated umwelt the way a real worm navigates the real one, we've captured something real, even if the underlying implementation differs.

What This Tests

It's a testbed for the embodiment hypothesis. If you build the umwelt and the simulated connectome still doesn't produce worm behavior, that tells you something important. Either:

1. The neural simulation is missing crucial details, or
2. There's something in biological implementation (glia, hormones, body physics) that you're still missing

This is still really hard. Defining "perfectly mirrors" for a creature whose experience you can't directly access requires reverse-engineering what features of reality the worm's sensory neurons actually encode. Which is itself a massive research project.

Why This Approach Might Actually Work

The critical insight: the umwelt is almost certainly compositional in a way intelligence might not be.

A worm's sensory world is basically a finite set of transducers:

• These 12 chemoreceptor types respond to these molecular classes
• These mechanoreceptors fire when the cuticle deforms by X amount
• These thermosensors encode temperature gradient direction

Each one is a relatively clean input-output mapping that evolution has already done the work of defining. You're not trying to figure out what matters. The worm's sensory apparatus already tells you what the worm "cares about." You just need to measure the transfer functions.

Contrast this with simulating the physical world: you'd need fluid dynamics for the bacterial soup it swims in, mechanical properties of agar substrates, diffusion equations for chemical gradients, thermal conductivity... and you'd need all of these at whatever resolution turns out to matter, which you don't know in advance.

The Research Program

So the path forward becomes:

1. Catalog every sensory neuron type in C. elegans (done, basically)
2. Characterize the stimulus-response function of each (partially done)
3. Build a virtual environment that delivers those stimuli in coherent spatiotemporal patterns
4. Close the loop with motor outputs affecting sensory inputs

Step 3 is the clever bit. You're not simulating physics. You're simulating what physics looks like to a worm. Which is a dramatically smaller problem.

Conclusion: The Path Forward

If we can't simulate a 302-neuron worm behaving in its natural environment, we have no business claiming we're close to AGI. But if we can, we'll have proven something profound: that we can build embodied, adaptive intelligence from first principles.

This won't give us AGI directly. But it will give us:

• A working testbed for theories of intelligence
• Proof that embodiment matters in ways we can measure
• A scalable methodology that might actually work for more complex systems
• A reality check on the "just scale transformers" approach

The path to AGI may not run through ever-larger language models. It may run through understanding how 302 neurons in a millimeter-long worm create genuine, adaptive, embodied intelligence.

And if we can't crack that nut, we're nowhere near cracking the 86-billion-neuron one.