Wanted to share a new version (much cleaner!) of a preprint on how connectivity structure shapes collective dynamics in nonlinear RNNs. Neural circuits have highly non-iid connectivity (e.g., rapidly decaying singular values, structured singular-vector overlaps), unlike classical random RNN models.
Connectivity structure and dynamics of nonlinear recurrent neural networks
Studies of the dynamics of nonlinear recurrent neural networks often assume independent and identically distributed couplings, but large-scale connectomics data indicate that biological neural circuit...
We introduce the "random-mode model," a random-matrix ensemble similar to an SVD that enables control of both the spectrum and mode overlaps. The key distinction from well-studied low-rank RNNs (Ostojic et al.) is that we use extensive rank scaling (number of modes scales with network size).
Aug 19, 2025 15:42This extensive rank scaling—in agreement with recent connectome data—enables modeling more sophisticated, higher-dimensional dynamics and computations—in agreement with recent large-scale neural recordings—compared to intensive-rank models that produce correspondingly low-dimensional activity.
Placing the random-matrix model we have defined into a nonlinear RNN, we compute single neuron (two-point functions) and collective (four-point functions) properties (such as the dimension of activity), doing the latter with a novel path-integral fluctuation calculation around the DMFT saddle.
(We previously did this for the iid case with a two-site cavity method [Clark et al, PRL 2023] that becomes unwieldy when applied to structured connectivity models.)
Connectivity structure can be invisible in single neurons but dramatically shape collective activity. Just two connectivity properties—eff coupling strength (local info) and eff rank (global info)—determine four-point functions, and thus dimensionality, despite high-dimensional nonlinear dynamics.
We treat limiting cases (where the dependance of dimensionality on connectivity properties becomes simple/interpretable), compare low-dim connectivity vs. firing-rate heterogeneity, and show how structured mode overlaps (demonstrated in the fly connectome) further shape collective activity.
Overall, we show a suite of tools that we think will be useful in linking neural-circuit structure→function. Crucially, these tools go beyond single-neuron activity to collective activity, and beyond finite-rank models (which are perhaps too low-dimensional) to high-rank, but not iid., connectivity.
Such high-dimensional, highly structured connectivity and dynamics (that, somehow, ultimately underlie task performance) are, I think, a main frontier of theoretical neuroscience research. This work constitutes a step in this direction.
(Link, again:
arxiv.org/abs/2409.01969)
