[PhD] Neural populations and guided learning on a neuromorphic chip

Supervision

BISCUIT team, Loria
Supervisor : Bernard Girau (HDR)
Bernard.Girau@loria.fr

Keywords

Neural networks, neural fields, neuromorphic chip, reinforcement learning

Description

Scientific context

The BISCUIT team studies computational paradigms where calculations are adaptive, distributed and decentralized, carried out by populations of simple computing units that communicate mainly with their close neighbors. These properties are compatible with the implementation of unsupervised – but not un-guided – self-organization principles to tackle difficult problems such as situated cognitive computation, autonomous robotics, adaptive allocation of computation resources, etc.

Among various alternatives to standard Von Neumann architectures, neuromorphic approaches currently benefit from the recent application successes of deep learning methods and the implication of major semiconductor companies through impressive neuromorphic chips such as IBM TrueNorth and the prototype chip from Intel called Loihi [7](https://www.intel.com/content/www/us/en/research/neuromorphic-computing.html).

The subject proposed here is linked to the definition and hardware simulation of populations of spiking neurons with heterogeneous and adaptive synaptic connections on a neuromorphic chip. Our priviledged models of neural populations are the DNF model (Dynamic neural fields) of the continuous interaction between excitatory and inhibitory neurons in a cortical population, and the SOM model of the self-organization within cortical columns. DNF have been successfully applied to visual memory, tracking, selection, scene exploration, human motion discrimination, etc. SOM have also been applied to a wide variety of vector quantization problems where their topological properties bring a significant added value, such as data visualization, image compression, novelty detection, etc. The computational requirements in terms of bandwidth and co-localized computation-memory needs make these models difficult to implement in real-world scenarios on conventional computers when real-time or low-consumption constraints exist. It is thus tempting to implement them in specialized hardware. Several attempts already exist on neuromorphic chips, using classical DNF with homogeneous and pre-computed connection kernels. We propose to explore how on-chip learning and neural populations can co-exist on the recent neuromorphic chip Loihi, taking advantage of a rewarded version of the spike-timing dependent plasticity (STDP, [9]) available since this chip incorporates reward spikes as well as STDP learning mechanisms.

Detailed description

The goal of this thesis is to adapt different models of neural populations to digital neuromorphic chips, and more especially to the Loihi architecture, while taking into account rewarded learning in the computation of lateral connection weights. This work can be divided into two complementary tasks.

1- Definition and analysis of Loihi-compliant models of neural populations, including DNF and SOM. These models must use the kind of spiking computation that can be implemented in the Loihi neuromorphic cores. In the field of vision, we have already shown that spiking DNF are able to select and track centers of interest in visual scenes, despite the presence of a strong level of noise or distractors (cf [1]), and we have derived an FPGA implementation coupled with an event-based DVS camera in [2]. We also have defined a spiking version of SOM that uses a temporal coding [5]. In terms of communication of neurons within neuromorphic cores and between them, a first approach can rely on AER-based protocols. To ensure a better scalability, more cellular and distributed communication protocols must be explored. For DNF, we have already explored their combination with cellular computing properties, first showing that a randomized propagation of spikes was able to simulate the same kind of synaptic weights as in usual DNF even when neurons are only connected to their 4 immediate neighbors [3], then mixing this spiking and randomized approach with the bit-stream arithmetic principles to define compact spike-stream operators [4]. For SOM, we have defined a cellular version of their learning to fit a bio-inspired manycore architecture [6]. The combination of these cellular principles with communications on neuromorphic chips can be studied in the perspective of the integration of very large neural populations.

2- The Loihi chip features synaptic weight learning based on STDP. Though most DNF applications use predefined weights, we have developed multi-map DNF models that learn their lateral synaptic weights with respect to the visual task to perform. These models have not yet been adapted to spiking computations, contrary to our spiking model of SOM that also uses a kind of STDP learning, but with computational needs that are quite different from the resources available on neuromorphic chips such as Loihi. The on-chip learning capability of the Loihi chip motivates the definition of efficient algorithms to learn the synaptic weights of spiking DNF and SOM by means of STDP and to map them on neuromorphic chips. Furthermore, our approach of unsupervised but not unguided learning fits the presence of specific reward spikes in the Loihi chip. Therefore, we propose the use of mechanisms at the crossroads of neuro-inspired calculation and reinforcement learning (or RL, [10]) in order to guide the processes of self-organization in DNF and SOM. The classical RL algorithms are mainly dedicated to discrete and centralized approaches that are not very compatible with our computation paradigms. To make them compatible with neuromorphic resources, we propose to modulate STDP with RL. Several studies have already been carried out in this direction (see, for example, [8]), but these works are still few in number and limited to rewarded versions of STDP without a real notion of reinforcement to improve long-term rewards.  In this part of the thesis, we will explore the capacity of existing rewarded-STDP models to learn our neural population models, then propose adaptations of these algorithms compatible with the constraints imposed by neuromorphic processors, and finally adapt these algorithms so as to rely on localized rewards and localized computations ensuring a global reinforcement learning.

Skills required

In addition to advanced master’s level computer skills, we expect solid foundations on the associated mathematical concepts (in particular probabilities and differential equations). The candidate should have some appetence for artificial intelligence and distributed numerical computation. Adequate knowledge of digital hardware design will be valued, as well as experience in software design. The candidate must fluently speak English and/or French.

Bibliography

[1] R. Vazquez, B. Girau, J.-C. Quinton. Visual attention using spiking neural maps. International Joint Conference on Neural Networks IJCNN 2011, Ali Minai, Hava Siegelmann, Jul 2011, San José, United States.

[2] B. Chappet de Vangel, C. Torres-Huitzil, B. Girau. Event based visual attention with dynamic neural field on FPGA. International Conference on Distributed Smart Camera, Sep 2016, Paris, France.

[3] B. Chappet de Vangel, C. Torres-Huitzil, B. Girau. Randomly spiking dynamic neural fields. Journal of Emerging Technologies in Computing Systems, ACM, 2014.

[4] B. Chappet de Vangel, C. Torres-Huitzil, B. Girau. Stochastic and Asynchronous Spiking Dynamic Neural Fields. International Joint Conference on Neural Networks (IJCNN 2015), Jul 2015, Killarney, Ireland.

[5] A. Fois and B. Girau. A Spiking Neural Architecture for Vector Quantization and Clustering. In: ICONIP 2020, 27th International Conference on Neural Information Processing.

[6] B. Girau and A. Upegui. Cellular Self-Organising Maps – CSOM. In: WSOM’19 – 13th International Workshop on Self-Organizing Maps and Learning Vector Quantization, June 2019.

[7] Davies, M., Srinivasa, N., Lin, T., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y., Wild, A., Yang, Y., and Wang, H. (2018). Loihi : A
neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1) :82–99.

[8] Florian, R. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation, 19(6):1468–1502.

[9] Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science, 275(5297) :213–215.

[10] Sutton, R. and Barto, A. (1998). Reinforcement Learning. Bradford Book, MIT Press, Cambridge, MA.

How to apply

Deadline: May 20th, 2021 (Midnight Paris time)
Applications are to be sent as soon as possible.

Send a file with the following components.

  1. Your CV;
  2. A cover/motivation letter describing your interest in this topic;
  3. A short (max one page) description of your Master thesis (or equivalent) or of the work in progress if not yet completed;
  4. Your degree certificates and transcripts for Bachelor and Master (or the last 5 years);

In addition, a recommendation letter from the person who supervise(s|d) your Master thesis (or research project or internship) is welcome.

Logo d'Inria