New #Preprint Alert!! 🤖 🧠 🧪
What if we could train neural cellular automata to develop continuous universal computation through gradient descent ?! We have started to chart a path toward this goal in our new preprint:
arXiv: arxiv.org/abs/2505.13058
Blog: gabrielbena.github.io/blog/2025/be...
🧵⬇️
Here's the gist: Traditional CAs (think Conway's Game of Life) have been mathematically proven Turing-complete... but designing them is HARD. You have to hand-craft rules, at the cost of arduous efforts.
What if instead we could just... train them to compute, offloading the burden?
Enter #NCA !
For those of you who've missed it, quick NCA primer:
- Traditional cellular automata = hand-crafted rules (like Conway's Game of Life).
- Neural Cellular Automata = local rule learned by a neural network through gradient descent!
distill.pub/2020/growing...
We propose a novel framework that disentangles the concepts of “hardware” and “state” within the NCA. For us:
- Rules = "Physics" dictating state transitions.
- Hardware = Immutable + heterogeneous scaffold guiding the CA behaviour.
- State = Dynamic physical & computational substrate.
Think of it like having a computing substrate:
- Some universal laws of physics apply to every unit of a motherboard / of a brain.
- These units are (usually) setup in a fixed, meaningful manner...
- But their evolving state (electrical charges / neurochemical patterns) govern the computation.
But how do we "instruct" the NCA what to do, what task to perform, on which data ? Basically, how do we interface with this dynamical substrate to "make" it do interesting computation ?
This is the role of the hardware ! This acts as a translation between human intent, and the dynamical substrate.
Through this framework, we are able to successfully train on a variety of computational primitives of matrix arithmetics.
Here is an example of the NCA performing Matrix Translation + Rotation directly in its computational state (and, by design, only using local interactions to do so) !
But here's where it gets REALLY wild...
We didn't just train on computational primitives... We then used our pre-trained NCA to emulate a small neural network and solve MNIST digit classification!
The entire neural network "lives" inside the CA state space!
More on the MNIST demo: We pre-train a linear classifier, decompose the 784×10 matrix multiplication into smaller blocks, and let the NCA process them in PARALLEL!
Emulated accuracy: 60% (vs 84%), not perfect due to error accumulation, but it WORKS! This is a neural network running inside a CA! 🤯
Btw, this isn't just academic curiosity. We're talking about:
🔸 Analogue computers that could be more efficient than digital ones (without the need to revert to binary-level operations).
🔸 #Neuromorphic computing that mimics how brains actually work.
🔸 Bypassing the von Neumann bottleneck ?
Our approach also enables task composition, meaning we can chain operations together!
Example: Distribute matrix → Multiply → Rotate → Return to original position
It's like programming, but the "execution" is continuous dynamics! We're building a neural compiler!
I quite like this idea of a compiler ! Think of it like having dual timescale:
- FAST: State / neuronal dynamics (computation happens)
- SLOW: Hardware reconfiguration (program flow)
This separation mirrors classical computer architecture but within a continuous, differentiable substrate !
Taking it even further: We're developing a graph-based "Hardware Meta-Network"!
Users define tasks as intuitive graphs (nodes = regions, edges = operations), and a GNN + coordinate-MLP generates the hardware configuration!
It's literally a compiler from human intent → NCA computation! 🤖
In conclusion: continuous cellular automata could be universal computers when trained right. This might change how we should think about:
- What can compute ?
- How to design computers ?
- The future of efficient AI hardware.
🚀 Let's train physics-based computers !🚀
