[PhD] Self-organizing maps on neuromorphic circuit
Supervision
BISCUIT team, Loria
Supervisor : Bernard Girau (HDR)
Bernard.Girau@loria.fr
Keywords
Neural networks, self-organizing maps, neuromorphic chip
Description
Scientific background
Among various alternatives to standard Von Neumann architectures, neuromorphic approaches currently benefit from the recent application successes of deep learning methods and the implication of major semiconductor companies through impressive neuromorphic chips such as IBM TrueNorth and the Intel chips Loihi and Loihi 2 ([1,2]). The Loihi 2 chip, for example, integrates one million spiking neurons and 120 millions programmable synaptic connections. The emergence of these neuromorphic chips is closely linked to the advantages offered by the so-called third generation neurons for hardware implantations. These neurons communicate temporally through impulses called spikes, directly inspired by the action potentials exchanged by biological neurons, thus allowing information to be transmitted and processed on the fly asynchronously.
In the same bio-inspired approach, event cameras are growing in popularity [3]. Event cameras like DVS (Dynamic Vision Sensor) work similarly to the retina by transmitting information as pulses called spikes only when a local change in brightness – at the pixel level – is detected. This asynchronous processing of visual information brings great advantages: 1) a sampling rate nearly a million times faster than with standard cameras, 2) a latency of one microsecond and 3) a dynamic range of 130 decibels (standard cameras only have 60 dB). All this for a significantly lower energy consumption than with standard cameras.
Detailed description
Our research team is studying different models of bio-inspired neural networks, some of which being designed with the perspective of a final implantation on a neuromorphic circuit. In particular, we have defined in [4] a spiking version of self-organizing maps (SOM), whose learning is obtained by a STDP rule (Spike-Timing Dependent Plasticity, [5]) that we have defined in order to encode the information in time of the spikes and not in their frequency. Like Kohonen maps ([6]), a known model of self-organization inspired by the cortex, our spiking SOMs perform unsupervised vector quantization of data in which prototypes organize themselves according to predefined neighborhood rules. Spiking self-organizing maps are at the heart of this thesis. The objective is to adapt and enrich them in order to exploit them in a neuromorphic context, in particular related to the Intel Loihi architecture. Different aspects need to be addressed.
1- Taking into account direct neuromorphic stimuli: the temporal coding used in our current SOM model translates real numbers into spikes. With the advent of event cameras, it is necessary to take into account stimuli that are directly delivered in the form of uncontrolled spikes. These cameras being above all sensitive to movements, we can for example study how to extract the optical flow through the temporal coding and learning that we have defined.
2- Even though the Loihi chip includes a programmable STDP learning of each synaptic weight, the rules that our model uses cannot be directly expressed in the Loihi computational model. Consequently, many adaptations will have to be proposed to achieve the implementation of spiking SOMs on such neuromorphic circuit, with the final objective of coupling it directly with event cameras.
Various event cameras as well as an access to the Loihi and Loihi 2 development platforms (through an agreement with Intel) will be available for this research project.
Skills required
The candidate should have the equivalent of a Master in Computer Science, preferably in a specialty related to artificial intelligence and/or distributed numerical computation. Adequate knowledge of digital hardware design will be assessed, as well as a solid experience in software design. Any work already done in the field of neuromorphic computing will be an important asset. The candidate must be fluent in English and/or French.
Bibliography
[1] M. Davies, N. Srinivasa, T.-H. Lin, Tsung-Han, G. Chinya, Y. Cao, S.H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, Y. Liao, C.-K. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul, J. Tse, G. Venkataramanan, Y.H. Weng,A. Wild, Y. Yang and H. Wang. Loihi : A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro, 38 (1), 2018.
[2] https://www.intel.com/content/www/us/en/research/neuromorphic-computing.html
[3] G. Gallego, T. Delbruck, G. Orchard, C. Bartolozzi, B. Taba and A. Censi, S. Leutenegger, A. Davison, J. Conradt, K. Daniilidis, D. Scaramuzza. Event-Based Vision : A Survey. IIEEE Transactions on Pattern Analysis and Machine Intelligence , 2020.
[4] A. Fois and B. Girau. A Spiking Neural Architecture for Vector Quantization and Clustering. 27th International Conference on Neural Information Processing (ICONIP), 2020.
[5] Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy ¨ by coincidence of postsynaptic APs and EPSPs. Science, 275(5297) :213–215.
[6] T. Kohonen. The self-organizing map. Neurocomputing, 21(1-3), 1998.
