Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

  • Spiking neural networks (SNNs) are not yet efficient and robust enough for control.
  • Biological agents use sparse and irregular spiking patterns for efficient and robust control.
  • Most artificial SNNs used for control have dense and regular activity patterns.
  • Spike Coding Networks (SCNs) offer a fully analytical solution for implementing dynamical systems in recurrent SNNs.
  • SCN theory is extended to incorporate closed-form optimal estimation and control.
  • Robust spiking control of simulated spring-mass-damper and cart-pole systems is demonstrated.
  • No learning or optimization is needed, offering opportunities for fast and efficient on-chip spiking controllers.

Paper Content

I. introduction

  • Brain and behavior are linked
  • Neurons control body through sparse spiking activity
  • SNNs are improving but still have open challenges
  • Analytical solutions are desirable for control
  • Biological spiking codes are irregular and sparse
  • SCN theory follows predictive coding principle
  • SCN theory can be used to estimate and control external systems
  • Proposed method combines optimal control and SCN theory

A. contribution

  • Formalize link between optimal control and SCN framework
  • Derive spiking equivalent of LQG control problem
  • Proposed networks accurately estimate and control well-known dynamical systems
  • Preserve irregular and sparse spiking patterns and robustness to neuron death
  • Relevant for industrial applications, robotics and machine intelligence
  • Non-exhaustive review of research related to SNNs
  • Work based on paper showing how linear dynamical systems can be implemented in an SCN
  • Follow-up papers on extending framework for general computations
  • Paper on extending spike coding networks with learning rules for connectivity weights
  • Paper proposing analytical SCN-inspired framework for control
  • Our approach takes inspiration from SCN literature to provide unified mathematical framework for optimal control in SNNs

Ii. methods

A. the spiking control problem

  • System state x(t) needs to be controlled with a spiking signal (s(t))
  • Network of N leaky integrate-and-fire (LIF) neurons is defined by voltage dynamics
  • Network is provided with incomplete measurements of system state y(t) and target state z(t)
  • Network has to generate control signal u(t)

B. spike-coding network theory

  • SCN framework can be used to implement linear dynamical systems in an SNN
  • SCN theory defines how a recurrent SNN can optimally track an external signal
  • SCN theory defines an SNN which implements a linear dynamical system
  • Network takes in both the current state and its dynamics as inputs
  • Recurrent connections make sure that spiking in the network is coordinated
  • Slow connections implement the desired dynamics of the linear dynamical system

Iii. closed-form optimal estimation and control

  • SCNs can be used to implement dynamical systems
  • A new mathematical framework allows for classical control in SNNs
  • The resulting recurrent SNN can act as a state-estimator and controller for an external dynamical system
  • The controller is a linear-quadratic-Gaussian spiking controller with efficient spiking patterns coding

A. the classical control problem

  • K-dimensional linear dynamical system in statespace representation
  • System matrix A conveys dynamics of system
  • Input vector u can control system
  • Internal disturbances η d and sensor noise η n given by zero-mean Gaussian process
  • Measurement vector y, system matrix A, input matrix B, input vector u, observability matrix C, and co-variances Σ d and Σ n are generally assumed as known
  • Goal is to control x to reference state z
  • Estimation and optimal control problems must be solved to generate correct output of SNN

B. optimal state estimation with scns

  • Kalman filter can provide full-state estimates given noisy and incomplete measurements in linear systems
  • Kalman filter gain matrix is applied to an error between the observations of the dynamical system and the Kalman filter’s own internal estimate
  • Kalman filter gain matrix can be found by solving an algebraic Riccati equation
  • SCN framework can be used to implement the optimal filtering with slow and fast connectivity, input connections, recurrent and feed-forward “Kalman filter” connections

C. optimal control with scns

  • Optimal state-estimation can be extended to optimal control
  • Linear-Quadratic Regulator (LQR) controller produces optimal signal to minimize error between state and target
  • Kalman filter and LQR controller can be combined to control partially observable and/or noisy dynamical system
  • Spiking network can be used to implement this combination
  • Reference signal is encoded into the network
  • Output of the internal LQR controller can be read-out from neural activities
  • Spiking network can track, replicate and control an external dynamical system

Iv. results

  • Evaluated performance of proposed mathematical framework
  • Analyzed properties of spiking patterns for internal robustness
  • Applied networks to two standard dynamical systems

A. spring-mass-damper (smd) system

  • Compared SCN estimation and control with non-spiking counterparts for spring-mass-damper system
  • Evaluated system robustness against input-noise, neural silencing and external perturbations
  • Estimation: SCN estimates both dynamical variables of real dynamical system with high accuracy
  • Control: SCN computes optimal u to make state of simulated dynamical system converge towards reference signal
  • Controller robustness: SCN able to keep system controlled when facing severe neural silencing
  • SCN shows signs of neurons ’taking over’ spiking of removed neurons
  • Evaluated effects of different parameter-values for sensor noise covariance and sudden pulse of force on controller error
  • Larger values of sensor noise covariance and pulse of force increase controller error
  • SCN controller quickly tries to correct for displacement of mass, bringing it back to reference position

B. cartpole system

  • Evaluated proposed SCN controller on nonlinear cartpole system
  • Cartpole has 4 dynamical variables
  • System has internal disturbance and outputs partially observable state
  • Linearized dynamics used to produce control signal
  • Ideal and spiking controllers followed reference signal and kept pole upright
  • Spiking activity increased when reference signal demanded change

V. conclusion

  • Closed-form control solutions exist for fully identified systems.
  • Spiking neural networks can be used to implement these solutions.
  • SCN theory can be extended to allow optimal estimation and control.

Vi. acknowledgements

  • Discussed control and spiking networks with Marcel van Gerven, Bodo Rückauer, Justus Hübotter, Christian Machens, William Podlaski, and Michele Nardin
  • Thankful for helpful discussions

Vii. appendix: detailed derivation of network

Connectivity

  • SCN theory can be used to implement a continuous Kalman filter and LQR control in an SNN.
  • Autoencoding network takes external input x and its derivative ẋ and can accurately track this signal with the read-out Dr.
  • Replace the derivative of the input with desired dynamics and estimate current state from network read-out.
  • If f (x, u) is purely linear, this can be done through a set of slow connections.
  • Nonlinear cases are not considered in this paper, but an analytical solution exists.

A. estimation

  • Kalman filter can be defined using equation 7
  • Voltage dynamics can be expressed as equation 8

B. control + estimation

  • Equation 10 is used to define a Kalman filter with LQR control.
  • The target state z needs to be represented in the network.