(2/26) Most neural network models treat synapses as static, but in actual neural circuits, neuronal and synaptic dynamics are tightly coupled. We show that this coupling enables a novel form of dynamic memory.

(3/26) We study a nonlinear recurrent-network model, introduced by me and Abbott (PRX, 2024), where synaptic weights fluctuate around fixed random baselines via ongoing Hebbian plasticity. The static and plastic components of the weight matrix are denoted J and A(t), with J_ij ~ N(0, g²/N).

(4/26) Neurons and synapses fluctuate on comparable timescales. Crucially, plasticity does not erase prior connectivity but sculpts existing structure through low-rank weight changes.

(5/26) Following Rajan, Abbott, and Sompolinsky (PRE, 2010), we drive these plastic networks with oscillatory inputs, with random phases across neurons, for t<0. At t=0, we halt stimulation. The setup is schematized here:

(6/26) During stimulation, we find different dynamic regimes depending on input amplitude I and plasticity strength k, including regimes where the input-driven component coexists with intrinsically generated activity driven by J, as well as regimes of full input entrainment.

(7/26) The most surprising phenomenology happens after input cessation at t≥0: neurons continue oscillating long after stimulation ends, with lifetimes exceeding any intrinsic system timescale, including the plasticity forgetting timescale, by an order of magnitude (panel iii).

(8/26) These "persistent oscillations" require ongoing plasticity, exemplifying a self-renewing, coupled neuronal-synaptic process. In particular, persistent oscillations are thwarted if we prevent neurons from influencing synapses (i.e., if the closed neuronal-synaptic loop is opened) for t≥0.

(9/26) Persistent oscillations occur in an "intermediate" dynamic regime where activity expresses features of both external inputs and recurrent connectivity. In particular, too-strong inputs lead to neuronal activity being dominated by the input alone, preventing persistent oscillations.

(10/26) The frequency of persistent oscillations tracks the input frequency (within a preferred band). That is, faster inputs generally produce faster persistent oscillations. Thus, the autonomous dynamics reflect the temporal structure of previously experienced stimuli, indicating a dynamic memory!

(11/26) We now turn to mechanistic understanding. Persistent oscillations occur when J + A(t) has complex-conjugate outlier eigenvalues at stimulation offset (t=0). In particular, while networks in the persistent-oscillation regime develop complex outliers, other regimes develop only real outliers.

(12/26) The emergence of these complex outliers is surprising! Hebbian plasticity produces symmetric weight updates where A(t) = A(t)ᵀ, and symmetric matrices have only real eigenvalues. Moreover, adding a fixed low-rank symmetric matrix to a random J can only create real outliers at large N.

(13/26) Thus, to create complex outliers, A(t) must become correlated with J. This can happen only if activity reflects both inputs and recurrence, explaining the significance of being in the "intermediate" regime. Indeed, A(t) aligns to eigenvector subspaces of J associated w/ complex eigenvalues.

(14/26) To understand this alignment mechanism, we consider a toy scenario involving a "target" matrix  = 2α Re{ψψ†} where ψ is an eigenvector of J with complex eigenvalue η and α is a real scalar (thus,  is real, symmetric, and rank-two).

(15/26) We show that, at large N, adding  to J pulls eigenvalue η to λ = η + α, generating complex-conjugate outliers that can drive oscillations. Furthermore, A(0) ≈  can be generated using oscillatory inputs whose single-neuron phases are chosen based on the eigenvector ψ.

(16/26) This leads to a dynamic analog of Hopfield networks: rather than evoking static patterns by aligning inputs to symmetric-connectivity eigenstructure, we evoke dynamic patterns by aligning inputs to asymmetric-connectivity eigenstructure.

(17/26) Let us now return to the full, random-phase input case. We approximate A(0) ≈ 2α Re{ΨΨ†} where Ψ = (νI − J)⁻¹eⁱᶿ. Here, eⁱᶿ is a vector containing input phases θᵢ for each neuron; ν is a complex scalar that depends on the system parameters; and α = kI²/4.

(18/26) Using this approximation, we derive exact large-N expressions for outlier eigenvalues λ = g²ν/|ν|² + α/(|ν|² - g²). This correctly predicts the full phenomenology of persistent oscillations, including amplitude and frequency dependence, preferred frequency bands, and regime transitions.

(19/26) Furthermore, we show that, while Ψ is not in general an eigenvector of J, it becomes an increasingly good approximate eigenvector as ν → g⁺. Thus, the mechanism is essentially the same as in the targeted case.

(20/26) In sum, we have arrived at a conceptual understanding of, and analytical solution to, the behavior of coupled neuronal-synaptic dynamics in a nonlinear, input-driven recurrent network. In particular, we have shown that this behavior enables a useful computational function: dynamic memory.

(21/26) What about experimental links? Many working-memory (WM) studies report complex dynamic activity following stim. cessation, not aligning neatly with "sustained firing" WM theories. Our results suggest such activity could be generated by Hebbian plasticity, also underlying canonical WM models.

(22/26) Furthermore, studies report persistent oscillations following periodic stimuli & phase-locking to LFP oscillations during WM tasks, interpreted as evidence for intrinsic oscillatory circuitry. Our results suggest this may arise via ongoing plasticity, without preexisting circuit structure.

(23/26) Some concluding thoughts. In many models of synaptic plasticity-based learning, weight updates simply overwrite existing connectivity. Our model points out that plasticity can instead dramatically shape dynamics by manipulating the dynamic reservoir provided by static backbone connectivity.

(24/26) This mechanism is evocative of experimental findings in motor cortex and sensory areas that reveal apparent constraints on neural activity patterns during learning (e.g., from Yu, Batista, Chase, et al.).

(25/26) This work emphasizes that understanding memory-related neural activity requires modeling synaptic and neuronal dynamics together. Separating these processes, while convenient, can obscure circuit functions. Coupling enables new forms of computation beyond what either process achieves alone.