Chargement Évènements

« Tous les Évènements

Soutenance de thèse : Raphaël Bagat (Multispeech)

25 novembre 2026 @ 9:30 am - 11:30 am

Raphaël Bagat, doctorant dans l’équipe Multispeech, soutiendra sa thèse le 25 novembre à 9 heures 30 en salle C005.

Sa présentation sera en anglais.

Sa thèse est intitulée « Domain Adaptation for Air Traffic Control Automatic Speech Recognition with Limited Transcribed Data« .
Defense committee:
Reviewer: Yannick Estève, Professor, Avignon Université, France
Reviewer: Benjamin Lecouteux, Professor, Université Grenoble Alpes, France
Examiner: Marie Tahon, Professor, Le Mans Université, France
Examiner: Dietrich Klakow, Professor, Université de la Sarre, Germany
Supervisor: Irina Illina, Associate Professor, Université de Lorraine, France
Co-supervisor: Emmanuel Vincent, Senior Research Scientist, Centre Inria de l’Université de Lorraine, France
Guest: Junichi Yamagishi, Professor, National Institute of Informatics, Japan
Guest: Raphaël Duroselle, Ph.D., Inria Défense et Sécurité, France
Abstract:
Recent large-scale pre-trained automatic speech recognition (ASR) systems achieve strong performance across a wide range of conditions, yet their accuracy can substantially degrade when deployed in specialized domains that differ from their training data, such as the Air Traffic Control (ATC) domain. This thesis investigates domain adaptation strategies for air traffic control ASR, where transcribed data are limited.

One of the important limitations in ASR adaptation is the high cost of human supervision. Transcribing speech requires domain expertise. ATC communications present acoustic and linguistic variability, including non-native accents, diverse operational roles, noise, and bandwidth-limited radio channels. Collecting sufficient labeled data to cover all these factors is impractical.

Since different non-native accents exhibit distinct characteristics, pooling them into a single adaptation may fail to capture accent-specific patterns, while maintaining a separate model for each accent does not scale. We propose Mixture of Accent-Specific LoRAs, which combines accent-specific low-rank experts within a shared model. Our results show that this specialization improves multi-accent speech recognition and remains beneficial even when the speaker’s accent is unknown at inference.

While self-supervised learning with untranscribed data is commonly used to pre-train speech encoders before supervised training, exploiting it to adapt an already supervised encoder-decoder model, such as Whisper, is less straightforward because modifying the encoder may disrupt its compatibility with the pre-trained decoder. To address this issue, we propose BEST-RQ Encoder Adaptation with Re-training and Distillation, an adaptation method combining a self-supervised objective with knowledge distillation. BEARD adapts Whisper’s encoder while preserving its compatibility with the pre-trained decoder. BEARD significantly improves supervised adaptation.

The third contribution exploits generative data augmentation. Since the generation of additional speech data is only beneficial when the synthetic data faithfully capture the characteristics of the target domain, it is essential to account for both speaker variability and acoustic conditions. We propose a generative augmentation framework combining text-to-speech, voice conversion, accent conversion and an ATC acoustic simulation. Moreover, we introduce a novel controllable L1-to-L2 accent conversion system. We show that synthetic speech provides the greatest benefits when it reproduces accent diversity and the acoustic characteristics of the target radio channel.

Linguistic knowledge can be exploited when facing acoustic ambiguity. External language models provide a way of introducing an additional linguistic prior during decoding. We show that external LLMs improve recognition on general-domain speech through reranking and shallow fusion. Then, we study their use for ATC speech recognition.

Since ATC communications provide contextual cues such as callsigns and preceding dialogue, this information can help recognize the current utterance. We leverage Whisper’s native prompting mechanism to provide domain-specific information at inference. We show that prompting can successfully exploit contextual information such as callsigns and dialogue history.

Finally, we combine two complementary adaptation strategies. BEARD exploits untranscribed acoustic information during adaptation while prompting exploits contextual information at inference. Their combination allows us to investigate whether these two sources of information are complementary. By combining self-supervised encoder adaptation with contextual prompting, we achieve the best recognition performance on ATC data reported in this thesis.

Détails

  • Date : 25 novembre 2026
  • Heure :
    9:30 am - 11:30 am
  • Catégorie d’Évènement:

Lieu

  • C005