[PhD 2022] Toward a decentralized neural architecture of self-organizing maps

Context

Unconventional computing for robotics

The BISCUIT (Bio-Inspired Situated Cellular and Unconventional Information Technology) team from the Loria laboratory gathers researchers interested in unconventional computing, considering in particular bio-inspired artificial intelligence models, while focusing on “doing something with Spatialized and Decentralized Computing Populations (SDP)”, rather than to accurately model brain structures. This team is interested in exploring new computational paradigms to tackle challenging problems such as autonomous, situated cognitive computing etc. The relevance of these unconventional paradigms comes from the idea that the brain is more powerful than human technology to control autonomous agents (e.g. animals). Moreover, even if this is controversial (Jones, 2000), when more recently evolved architectures of the nervous system, such as the cortex, are taken into account, it appears that genetics encodes the anatomical development of fairly homogeneous systems, that are then adapted by learning when the animal interacts with the world (Miller et al., 2001; Ballard, 1986; Stavrinou et al., 2007). When they are considered as biological solutions to robotic problems, brains show that computing by assembling a large population of small elementary computational circuits (e.g. micro-columns in the in the cortex (Mountcastle, 1997)) is a robust and efficient way to control artificial agents. But, the reproduction of the modes and capacities of calculation of the brains being not yet reached, the researchers in computer science do not understand fully these phenomena.

Self-organization

Among all the features that can be transferred from biology to computer science, this PhD wishes to emphasize self-organization, following the founding approach of Kohonen (Kohonen, 1997). Kohonen was inspired by the biology of the cortex for his model of self-organizing maps (SOM), a model that is nowadays a classical artificial learning algorithm for unsupervised learning. Previous work of the team members also focuses on self-organization, with emphasis on a more specifically SPD approach (Ménard, 2006; Alecu, 2011; Lefort, 2012; Khouzam, 2014), that is not central in classical SOMs. The idea is to consider self-organizing SPD modules as elementary blocks of computation for multi-map architectures (Ménard and Frezza-Buet, 2005). The way in which multiple self-organizing modules should be connected remains a crucial problem of this approach, knowing that they can also be connected to themselves. They then become able to process sequences of data and, as a consequence, the temporal nature of the information (Khouzam and Frezza-Buet, 2013). Our previous works have shown two main limitations. They have only been applied to toy problems of “proof-of-concept” and they require a large amount of parallel computation (Gustedt et al., 2010), since the intrinsic mechanisms rely on large-scale populations of elementary computational units. This strongly restricts the possibility to explore architectures composed of many SPD modules and to hope for a real-time use, in autonomous robotics for example. More recently, a thesis which is ending in the team has opened the way on these questions, at the level of an intermediate approach which defines a methodology for the construction of complex architectures. The aim is to define elementary bricks and their exchanges in order to allow a scaling up. These self-organizing bricks form a dynamic system whose relaxation towards basins of attraction constitutes the setting up of a consensus (Gonnier et al., 2021; Gonnier et al., 2020). This methodology, which is promising for the moment, remains to be stabilized, to be experimented on temporal problems and to be extended beyond the toy approaches that have guided its validation.
This is the purpose of the proposed thesis.

Experiments

We propose to decline the experimental part of this thesis in a robotic context. This PhD is not a direct contribution to robotics, since the goal is to address SPD computing rather than to provide a robot with capabilities beyond those of the state of the art. Nevertheless, a challenge for this thesis will be to use a real robot as a validation platform. To do so, the CentralSupélec smartroom will be available : applications to drones (Parrots quadricopters), for which we have first results (Gonnier et al., 2021), or to wheel robots (Kheperas, turtlebots) can be easily accessible, using ROS.

Objectives

As already said, the self-organization modules have already been addressed within the team, focusing on the calculation of fine-grained populations. A recent turn has been taken towards more mesoscopic architectures based on SOMs adapted to the support of a consensus to direct self-organization processes. However, it remains today to build multi-map architectures with many components and to analyze their dynamic behavior. Multi-map approaches of self-organization have been proposed in the literature (Johnsson et al., 2009), as well as recurrent approaches for temporal processing (Voegtlin, 2002; Hagenbuchner et al., 2001), but the number of modules involved remains low : in the contributions where it is higher than one, it remains always less than three. The type of computations that can be performed by self-organizing architectures composed of a large number of modules remains to be investigated, even after the definition of the methodology introduced by the work in progress in the team.

Working conditions and skills expected

The PhD student will be hosted at Loria, a bi-located laboratory in Nancy and Metz (at the CentralSupelec campus). He or she will work on both sites, at his/her convenience, under the supervision of Hervé Frezza-Buet and Yann Boniface. Scientific collaborations with the other members of the team is expected, as well as more general scientific discussions and collaborations with other members of the laboratory. The expected duration of the PhD is three years. References to biology should be taken into account, a taste for innovation and multidisciplinary approaches is expected. Good programming skills are also required, as the tools we provide for the study are written in C++ and Python in particular.
The team will provide a set of programming tools, robotic platforms all the necessary human support for the technical aspects which will allow the PhD student to focus on the scientific questions. See for example the cxsom suite (https://github.com/HerveFrezza-Buet/cxsom).

Références

Alecu, L. (2011). Une approche neuro-dynamique de conception des processus d’auto-organisation. PhD thesis, Université Henri Poincaré – Nancy I.
Ballard, D. H. (1986). Cortical connections and parallel processing : Structure and function. Behavioral Brain Science, 9 :67–129.
Gonnier, N., Boniface, Y., and Frezza-Buet, H. (2020). Consensus Driven Self-Organization : Towards Non Hierarchical Multi-Map Architectures. In Communications in Computer and Information Science, Neural Information Processing, ICONIP 2020, pages 526–534.
Gonnier, N., Boniface, Y., and Frezza-Buet, H. (2021). Input Prediction Using Consensus Driven SOMs. In ISCMI 2021 :8th Intl. Conference on Soft Computing & Machine Intelligence.
Gustedt, J., Vialle, S., Frezza-Buet, H., Sitou, D. B., and Fressengeas, N. (2010). InterCell : a Software Suite for Rapid Prototyping and Parallel Execution of Fine Grained Applications. In PARA 2010 conference : State of the Art in Scientific and Parallel Computing.
Hagenbuchner, M., Tsoi, A. C., and Sperduti, A. (2001). A supervised self-organizing map for structured data. In Advances in Self-Organising Maps, pages 21–28.
Johnsson, M., Balkenius, C., and Hesslow, G. (2009). Associative self-organizing map. In proceedings of the International Joint Conference on Computational Intelligence (IJCCI), pages 363–370.
Jones, E. G. (2000). Microcolumns in the cerebral cortex. PNAS, 97(10) :5019–5021.
Khouzam, B. (2014). Neural networks as cellular computing models for temporal sequence processing. PhD thesis, Supélec.
Khouzam, B. and Frezza-Buet, H. (2013). Distributed recurrent self-organization for tracking the state of non-stationary partially observable dynamical systems. Biologically Inspired Cognitive Architectures, 3 :87–104.
Kohonen, T. (1997). Self Organizing Maps. Springer. Second Edition.
Lefort, M. (2012). Apprentissage spatial de corrélations multimodales par des mécanismes d’inspiration corticale. Theses, Université de Lorraine.
Ménard, O. (2006). Mécanismes d’inspiration corticale pour l’apprentissage et la représentation d’asservissements sensori-moteurs en robotique. PhD thesis, Université Henri Poincaré – Nancy I.
Ménard, O. and Frezza-Buet, H. (2005). Model of multi-modal cortical processing : Coherent learning in self-organizing modules. Neural Networks, 18(5-6) :646–655. extended version of Coherent learning in cortical maps : A generic approach, IJCNN’05.
Miller, K. D., Simons, D. J., and Pinto, D. J. (2001). Processing in layer 4 of the neocortical circuit : New insights from visual and somatosensory cortex. Current Opinion in Neurobiology, 11 :488–497.
Mountcastle, V. B. (1997). The columnar organization of the neocortex. Brain, 120 :701–722.
Stavrinou, M. L., Penna, S. D., Pizzella, V., Torquati, K., Cianflone, F., Franciotti, R., Bezerianos,
A., Romani, G. L., and Rossini, P. M. (2007). Temporal dynamics of plastic changes in human primary somatosensory cortex after finger webbing. Cerebral Cortex, 17(9) :2134–2142.
Voegtlin, T. (2002). Recursive self-organizing maps. Neural Networks, 15(8-9) :979–992.