Link to paper
The full paper is available here.
You can also find the paper on PapersWithCode here.
Abstract
- Spiking neural networks (SNNs) are not yet efficient and robust enough for control.
- Biological agents use sparse and irregular spiking patterns for efficient and robust control.
- Most artificial SNNs used for control have dense and regular activity patterns.
- Spike Coding Networks (SCNs) offer a fully analytical solution for implementing dynamical systems in recurrent SNNs.
- SCN theory is extended to incorporate closed-form optimal estimation and control.
- Robust spiking control of simulated spring-mass-damper and cart-pole systems is demonstrated.
- No learning or optimization is needed, offering opportunities for fast and efficient on-chip spiking controllers.
Paper Content
I. introduction
- Brain and behavior are linked
- Neurons control body through sparse spiking activity
- SNNs are improving but still have open challenges
- Analytical solutions are desirable for control
- Biological spiking codes are irregular and sparse
- SCN theory follows predictive coding principle
- SCN theory can be used to estimate and control external systems
- Proposed method combines optimal control and SCN theory
A. contribution
- Formalize link between optimal control and SCN framework
- Derive spiking equivalent of LQG control problem
- Proposed networks accurately estimate and control well-known dynamical systems
- Preserve irregular and sparse spiking patterns and robustness to neuron death
- Relevant for industrial applications, robotics and machine intelligence
B. related scn literature
- Non-exhaustive review of research related to SNNs
- Work based on paper showing how linear dynamical systems can be implemented in an SCN
- Follow-up papers on extending framework for general computations
- Paper on extending spike coding networks with learning rules for connectivity weights
- Paper proposing analytical SCN-inspired framework for control
- Our approach takes inspiration from SCN literature to provide unified mathematical framework for optimal control in SNNs
Ii. methods
A. the spiking control problem
- System state x(t) needs to be controlled with a spiking signal (s(t))
- Network of N leaky integrate-and-fire (LIF) neurons is defined by voltage dynamics
- Network is provided with incomplete measurements of system state y(t) and target state z(t)
- Network has to generate control signal u(t)
B. spike-coding network theory
- SCN framework can be used to implement linear dynamical systems in an SNN
- SCN theory defines how a recurrent SNN can optimally track an external signal
- SCN theory defines an SNN which implements a linear dynamical system
- Network takes in both the current state and its dynamics as inputs
- Recurrent connections make sure that spiking in the network is coordinated
- Slow connections implement the desired dynamics of the linear dynamical system
Iii. closed-form optimal estimation and control
- SCNs can be used to implement dynamical systems
- A new mathematical framework allows for classical control in SNNs
- The resulting recurrent SNN can act as a state-estimator and controller for an external dynamical system
- The controller is a linear-quadratic-Gaussian spiking controller with efficient spiking patterns coding
A. the classical control problem
- K-dimensional linear dynamical system in statespace representation
- System matrix A conveys dynamics of system
- Input vector u can control system
- Internal disturbances η d and sensor noise η n given by zero-mean Gaussian process
- Measurement vector y, system matrix A, input matrix B, input vector u, observability matrix C, and co-variances Σ d and Σ n are generally assumed as known
- Goal is to control x to reference state z
- Estimation and optimal control problems must be solved to generate correct output of SNN
B. optimal state estimation with scns
- Kalman filter can provide full-state estimates given noisy and incomplete measurements in linear systems
- Kalman filter gain matrix is applied to an error between the observations of the dynamical system and the Kalman filter’s own internal estimate
- Kalman filter gain matrix can be found by solving an algebraic Riccati equation
- SCN framework can be used to implement the optimal filtering with slow and fast connectivity, input connections, recurrent and feed-forward “Kalman filter” connections
C. optimal control with scns
- Optimal state-estimation can be extended to optimal control
- Linear-Quadratic Regulator (LQR) controller produces optimal signal to minimize error between state and target
- Kalman filter and LQR controller can be combined to control partially observable and/or noisy dynamical system
- Spiking network can be used to implement this combination
- Reference signal is encoded into the network
- Output of the internal LQR controller can be read-out from neural activities
- Spiking network can track, replicate and control an external dynamical system
Iv. results
- Evaluated performance of proposed mathematical framework
- Analyzed properties of spiking patterns for internal robustness
- Applied networks to two standard dynamical systems
A. spring-mass-damper (smd) system
- Compared SCN estimation and control with non-spiking counterparts for spring-mass-damper system
- Evaluated system robustness against input-noise, neural silencing and external perturbations
- Estimation: SCN estimates both dynamical variables of real dynamical system with high accuracy
- Control: SCN computes optimal u to make state of simulated dynamical system converge towards reference signal
- Controller robustness: SCN able to keep system controlled when facing severe neural silencing
- SCN shows signs of neurons ’taking over’ spiking of removed neurons
- Evaluated effects of different parameter-values for sensor noise covariance and sudden pulse of force on controller error
- Larger values of sensor noise covariance and pulse of force increase controller error
- SCN controller quickly tries to correct for displacement of mass, bringing it back to reference position
B. cartpole system
- Evaluated proposed SCN controller on nonlinear cartpole system
- Cartpole has 4 dynamical variables
- System has internal disturbance and outputs partially observable state
- Linearized dynamics used to produce control signal
- Ideal and spiking controllers followed reference signal and kept pole upright
- Spiking activity increased when reference signal demanded change
V. conclusion
- Closed-form control solutions exist for fully identified systems.
- Spiking neural networks can be used to implement these solutions.
- SCN theory can be extended to allow optimal estimation and control.
Vi. acknowledgements
- Discussed control and spiking networks with Marcel van Gerven, Bodo Rückauer, Justus Hübotter, Christian Machens, William Podlaski, and Michele Nardin
- Thankful for helpful discussions
Vii. appendix: detailed derivation of network
Connectivity
- SCN theory can be used to implement a continuous Kalman filter and LQR control in an SNN.
- Autoencoding network takes external input x and its derivative ẋ and can accurately track this signal with the read-out Dr.
- Replace the derivative of the input with desired dynamics and estimate current state from network read-out.
- If f (x, u) is purely linear, this can be done through a set of slow connections.
- Nonlinear cases are not considered in this paper, but an analytical solution exists.
A. estimation
- Kalman filter can be defined using equation 7
- Voltage dynamics can be expressed as equation 8
B. control + estimation
- Equation 10 is used to define a Kalman filter with LQR control.
- The target state z needs to be represented in the network.