[PhD] Self-organization and guided learning on neuromorphic circuit
BISCUIT team, Loria
Supervisor : Bernard Girau (HDR)
Neural networks, self-organizing maps, neuromorphic chip, reinforcement learning
Among various alternatives to standard Von Neumann architectures, neuromorphic approaches currently benefit from the recent application successes of deep learning methods and the implication of major semiconductor companies through impressive neuromorphic chips such as IBM TrueNorth and the Intel chips Loihi and Loihi 2 ([1,2]). With a system based on Loihi chips, up to 8 billions of spiking neurons can be implemented in a fully parallel way. The emergence of these neuromorphic chips is closely linked to the advantages offered by the so-called third generation neurons for hardware implantations. These neurons communicate temporally through impulses called spikes, directly inspired by the action potentials exchanged by biological neurons, thus allowing information to be transmitted and processed on the fly asynchronously. The Intel Loihi and Loihi 2 architectures more specifically attract our attention, in particular because of certain specificities such as a STDP-based programmable embedded learning (Spike-Timing Dependent Plasticity, ) and the presence of reward spikes.
Our research team is developing different models of bio-inspired neural networks, some of which being designed for a final implantation on a neuromorphic circuit. In particular, we have defined in  a spiking version of self-organizing maps (SOM), whose learning is obtained by a STDP rule that we have defined in order to encode the information in time of the spikes and not in their frequency. Like Kohonen maps (), a well-known model of self-organization inspired by the cortex, our spiking SOMs perform unsupervised vector quantization of data in which prototypes organize themselves according to predefined neighborhood rules. In addition, we are currently developing a reinforcement learning (RL) model using populations of spiking neurons playing the role of actor and critic, in a sufficiently simplified way to consider an implementation on the Loihi architecture.
The subject proposed here aims to deepen and combine these models in order to further explore the contribution of guided self-organization in the context of hardware neuromorphic computation.
Spiking self-organizing maps are at the heart of this thesis. The objective is to adapt and enrich them in order to exploit them in a neuromorphic context, in particular linked to the Intel Loihi architecture. Different aspects need to be addressed.
1- Taking into account direct neuromorphic stimuli: the temporal coding used in our current SOM model translates real numbers into spikes. With the advent of neuromorphic sensors such as event cameras (), it is necessary to take into account stimuli that are directly delivered in the form of uncontrolled spikes. These sensors, such as DVS (Dynamic Vision Sensor) cameras, indeed operate in a similar way to the retina by transmitting information in the form of a spike only when a local change in luminosity is detected at the level of a pixel. These cameras being above all sensitive to movements, we can for example study how the principles of spiking SOMs can extract the optical flow from them.
2- Even if the Loihi chip includes a programmable STDP learning of each synaptic weight, many adaptations will have to be proposed to achieve hardware implementations of spiking SOMs on this neuromorphic circuit, with the final objective of connecting them directly to event cameras. Various event cameras as well as an access to the Loihi and Loihi 2 development platforms (through an agreement with Intel) will be available for this research project.
3- Current SOM learning assumes that all stimuli are equal. To move towards a kind of situated learning in a real context, we propose to guide, modulate, motivate, self-organization according to an evaluation of the relevance of current stimuli thanks to a RL mechanism. Considering the targeted neuromorphic context, the proposed solution is to couple the STDP mechanisms used by our spiking SOM models and by the populations of neurons that play the roles of actor and critic in the neuromorphic AR that we are developing.
The candidate should have the equivalent of a Master in Computer Science, preferably in a specialty related to artificial intelligence and/or distributed numerical computation. Adequate knowledge of digital hardware design will be assessed, as well as a solid experience in software design. Any work already done in the field of neuromorphic computing will be an important asset. The candidate must be fluent in English and/or French.
 M. Davies, N. Srinivasa, T.-H. Lin, Tsung-Han, G. Chinya, Y. Cao, S.H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, Y. Liao, C.-K. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul, J. Tse, G. Venkataramanan, Y.H. Weng,A. Wild, Y. Yang and H. Wang. Loihi : A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro, 38 (1), 2018.
 Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy ¨ by coincidence of postsynaptic aps and epsps. Science, 275(5297) :213–215.
 A. Fois and B. Girau. A Spiking Neural Architecture for Vector Quantization and Clustering. 27th International Conference on Neural Information Processing (ICONIP), 2020.
 T. Kohonen. The self-organizing map. Neurocomputing, 21(1-3), 1998.
 Sutton, R. and Barto, A. (1998). Reinforcement Learning. Bradford Book, MIT Press, Cambridge, MA.
 G. Gallego, T. Delbruck, G. Orchard, C. Bartolozzi, B. Taba and A. Censi, S. Leutenegger, A. Davison, J. Conradt, K. Daniilidis, D. Scaramuzza. Event-Based Vision : A Survey. IIEEE Transactions on Pattern Analysis and Machine Intelligence , 2020.