Awesome Reservoir Computing

Awesome tutorials, papers, projects and tools for Reservoir Computing techniques like Echo State Networks (ESN).

Table of contents

• Introduction
• Tutorials
• Papers
• Tools
• Contributing

Introduction

• A Practical Guide to Applying Echo State Networks (2012) by Mantas Lukosevicius. Complete guide about ESNs, from theory to implementation.
• Echo State Network on Scholarpedia, by Herbert Jaeger. Generic introduction to Reservoir Computing from Echo State Networks.

Tutorials

Tuning Hyperparameters

• Hinaut & Trouvain (2021) Which Hype for My New Task? Hints and Random Search for Echo State Networks Hyperparameters. In International Conference on Artificial Neural Networks (pp. 83-97).

Founders

Early Reservoirs

• Dominey (1995) Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning. Biol. Cybernetics, Vol. 73, 265-274

• Buonomano & Merzenich (1995) Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties. Science 267, 1028-1030

Reservoir 2000's

• Jaeger (2001) The "echo state" approach to analysing and training recurrent neural networks. GMD Report 148, GMD - German National Research Institute for Computer Science

• Maass et al. (2002) Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11):2531-2560

Papers

Reviews

Theory of RC

Online learning

• Backpropagation-decorrelation: Steil, J. J. (2004). Backpropagation-decorrelation: Online recurrent learning with O(N) complexity. 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), 2, 843–848 vol.2.

• FORCE: Sussillo, D., & Abbott, L. F. (2009). Generating Coherent Patterns of Activity from Chaotic Neural Networks. Neuron, 63(4), 544–557.

• Reward-modulated Hebbian learning (3 factors rules): Hoerzer, G. M., Legenstein, R., & Maass, W. (2014). Emergence of Complex Computational Structures From Chaotic Neural Networks Through Reward-Modulated Hebbian Learning. Cerebral Cortex, 24(3), 677–690.

Intrinsic plasticity

• Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J. J., & Stroobandt, D. (2008). Improving reservoirs using intrinsic plasticity. Neurocomputing, 71(7), 1159–1171. https://doi.org/10.1016/j.neucom.2007.12.020

RC for neurosciences

• RC as model of prefrontal cortex activity: Enel, P., Procyk, E., Quilodran, R., & Dominey, P. F. (2016). Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex. PLOS Computational Biology, 12(6), e1004967.

• RC as model of the working memory: Strock, A., Hinaut, X., & Rougier, N. P. (2020). A Robust Model of Gated Working Memory. Neural Computation, 32(1), 153–181.

Recent papers

2021

• Pedrelli & Hinaut (2021). Hierarchical-task reservoir for online semantic analysis from continuous speech. IEEE Transactions on Neural Networks and Learning Systems.

• Manneschi et al. (2021). SpaRCe: Improved Learning of Reservoir Computing Systems through Sparse Representations. IEEE Transactions on Neural Networks and Learning Systems.

• Manneschi et al. (2021). Exploiting multiple timescales in hierarchical echo state networks. Frontiers in Applied Mathematics and Statistics, 6, 76.

2020

• Bianchi et al. (2020) Reservoir computing approaches for representation and classification of multivariate time series. IEEE transactions on neural networks and learning systems, 32(5), 2169-2179.

Tools

Contributing

Have anything in mind that you think is awesome and would fit in this list? Feel free to send a pull request.