The seminar “Unveiling Mixture of Experts for Multi-Sensor Fusion and Multi-Task Learning“, by Zongwei Wu, will take place on Friday, April 4, at 2 p.m., in room C103, at the Loria.
Abstract: Building toward a foundation visual model for real-world perception requires processing diverse sensor inputs while simultaneously handling multiple tasks. Achieving this demands efficient yet effective knowledge transfer across different domains—whether sensors or tasks. While recent Mixture of Experts (MoE) architectures, such as Mistral and DeepSeek, have demonstrated remarkable efficiency in large language models, they are not explicitly designed for cross-domain knowledge sharing.
In this talk, I will present our work on two novel MoE variations tailored for multi-sensor fusion and multi-task reasoning, respectively. For multi-sensor fusion, we introduce a Mixture of Modal Experts (MeME), a model that dynamically selects and integrates modality-specific experts to enhance cross-modal learning, improving robustness in uncertain environments. More importantly, MeME enables emergent alignment using only paired data.
For multi-task learning, we propose a Mixture of Complexity Experts (MoCE), an architecture that adapts expert selection based on task complexity. This design naturally transforms MoCE into a task-discriminative Learner, facilitating efficient multi-task learning with adaptive resource allocation.
Through these innovations, we show how MoE can go beyond efficiency, enabling effective cross-domain knowledge sharing for the next generation of perception models.
Bio: Zongwei Wu is a PostDoc Researcher at the Computer Vision Lab, University of Würzburg, Germany. He received his diplôme d’ingénieur from the University of Technology of Compiègne in 2019 and earned a Ph.D. in Computer Vision from Vibot EMR CNRS 6000, University of Burgundy, France in 2022. He was also a visiting scholar at CVL, ETH Zurich. His research focuses on multimodal models and multi-task reasoning for machine vision. He is a main organizer of the NTIRE workshop at CVPR 2024-2025 and serves as an Associate Editor for IEEE RA-L.
