The next NLP seminar, “Discrete Representation Models for Speech Data: Semantic vs. Acoustic Tokens” organized by Multispeech, Sémagramme, and Synalp teams, will take place at Loria on October 7, at 10 am, in room A008.

Speaker: Robin San Roman, Meta AI Paris & Multispeech.

Abstract: Discrete representation models for audio have emerged as a way to apply natural language processing techniques, such as language models (LMs), to the speech modality. We will discuss two families of representations: semantic tokens and acoustic tokens. Semantic tokens capture the meaning of speech at a high level, while acoustic tokens represent the raw audio signal. We will examine the pros and cons of each approach for speech representation, compression, and modelling. By understanding the strengths and limitations of these representations, we can develop approaches that bridge the gap between semantic and acoustic representations.