Neuroevolution is the branch of AI that trains neural networks not with gradient descent, but with evolutionary algorithms: populations of networks are mutated, evaluated, and selected over generations until a capable design emerges. It was once the main way to optimize both the weights and architectures of networks before backpropagation took over. Today, it is, I believe, making a quiet comeback in the LLM era. Labs like Sakana AI are using evolutionary search to discover model architectures, merge foundation models, and automate research workflows that pure fine-tuning struggles to explore. The idea is no longer to replace backpropagation, but to evolve the things gradient descent cannot easily optimize: topology, composition, and long-horizon strategies.

1. The bubble will pop

This should come as no surprise. Every time US labs claim the best AI model that no one else can match, an open-weights Chinese model shows up shortly after and beats it. The gap used to be months; now it is weeks. As I write this, Kimi K3 has just outperformed Fable on several benchmarks, even though Fable was released only a month earlier.

Scaling returns are diminishing: training data is running out, power grids are straining under training loads, and inference costs are rising while companies struggle to justify their API spending.

Kimi K3 and GLM-5 showed that American labs have no moat: they don't have a secret recipe, they don't have a better architecture, they don't have a better training algorithm. They use transformers like everyone else. You can't scale forever. They are just bigger, with diminishing returns from adding more data. The Chinese labs are catching up; should you be surprised?

This cannot last forever; nothing that cannot be sustained lasts forever. When the bubble pops, the AI landscape will be littered with the corpses of those labs. And just like with dinosaurs, the survivors will be the smaller ones.

2. Frugal will win

The current approach is becoming too expensive to sustain. Consider some numbers:

• OpenAI reportedly lost $21 billion last year on $13 billion in revenue.
• The US cloud industry has committed over a trillion dollars to AI infrastructure, yet none of the major AI labs are really profitable.
• A $100 Claude subscription gives you access to Claude Fable, but the equivalent token cost at list price would run into thousands of dollars.

Business models like this cannot last. Even if they somehow did, why pay hundreds of dollars for a model you do not own, with API costs that can become unsustainable tomorrow?

Of course, not everyone has a machine with 512 GB of RAM and four RTX 4090s. But you no longer need one. For example, with Colibri, you can run GLM 5.2, arguably a near-Fable-level model, on consumer hardware by using your SSD directly as memory. It is far slower, but I would rather have a model running at 0.2 tokens per second, 24/7, on my own machine than a subscription running at 20 tokens per second for $100 a month, asking me to add credits after 30 minutes of usage, and running on someone else’s server.

That was a detour into local AI, which matters, but it is not the full answer. Even with tools like Colibri, a hundreds-of-gigabyte model crawling along on consumer hardware is still unreachable for most of the population. When minutes are needed to generate an answer, no normie will ever use it, and few businesses would be sustainable. These systems are still monsters: they take months and millions of dollars to train, consume great amounts of power, and remain black boxes you cannot easily inspect and that adapt poorly to your needs.

Real frugality cannot just mean “run the same giants on a smaller machine.” It has to mean building models that need fewer parameters, less data, and less energy in the first place. This is where we either: (1) push the limits of training and inference optimization techniques, as well as compression, distillation, and pruning; (2) explore alternative approaches to architecture design; or (3) combine the strengths of all the small models the AI race produced. The latter is where neuroevolution becomes interesting again.

3. Optimize, discover or recombine

If "run the giants on smaller machines" isn't the answer, then what is? As I see it, there are three roads: push optimization and compression harder, rethink the architecture from scratch, or stop building bigger models at all and learn to combine the small ones we already have.

Solution 1: squeeze harder. This is the road China has been driving on; Kimi, DeepSeek, Moonshot, and Qwen have done a lot of impressive work on this front. Mixture-of-experts, speculative decoding, quantization-friendly training, distillation: they have pushed every one of these techniques to the edge. Yet, the most efficiency-obsessed labs on the planet still end up shipping trillion-parameter models...

Solution 2: new architecture. This is Yann LeCun's bet, and it might be the right one, 20 years from now. World models, JEPA-style architectures, are systems that learn from video the way children learn from sight. This is likely how we get AI that understands the world and doesn't tell you to go wash your car by walking to the garage... without your car. However, the first generation of JEPA will give us better robotics, not superintelligence. And it will take years to even get there. World models are anything but frugal. By LeCun's own calculation, a four-year-old has absorbed more data through vision than any LLM has ever seen in text. Video is vastly more token-hungry than text per unit of meaning. This doesn't mean it won't be useful, or even that it can never run on consumer hardware, but it might take a few years before the hardware allows you to run genuinely useful world models on your laptop or to provide cheap inference.

Solution 3: combine or evolve what we have. Few labs are exploring this path, but it is the one I believe will be the most fruitful in the next decade. The most notorious one on this journey is of course Sakana AI, based in Tokyo, founded by none other than some of the Transformer architecture's authors, David Ha and Llion Jones. Sakana AI is using evolutionary search to discover better algorithms, combine the strengths of multiple model architectures, merge foundation models, and automate research workflows that LLMs alone struggle to explore.

4. The comeback of Neuroevolution

In progress...

"The future of AI will not consist of a single, gigantic, all-knowing AI system that requires enormous energy to train, run, and maintain, but rather a vast collection of small AI systems" MIT Neuroevolution book.