Simon Rouard

Very happy to announce that my paper “Audio Conditioning for Music Generation via Discrete Bottleneck Features“ done with @honualx @adiyossLC @jadecopet and Axel Roebel has been accepted at ISMIR24. Paper: arxiv.org/abs/2407.12563 Sample: musicgenstyle.github.io Code: soon

@kyutai_labs Thank you @nmboffi, as we noticed that Lagrangian self-distillation worked much better than consistency for our TTS task.

🔓 Open source for everyone. Trained on 88k hours of public English data to ensure reproducibility. Check out the code and the technical breakdown here: kyutai.org/pocket-tts-tec…

We’re excited to introduce Pocket TTS: a 100M-parameter text-to-speech model with high-quality voice cloning that runs on your laptop—no GPU required. Open-source, lightweight, and incredibly fast. 🧵👇

Super happy that our work on Continuous Audio Language Models (arxiv.org/abs/2509.06926) led us to build an outstanding 100M TTS with voice cloning ability that runs on any laptop CPU.

Gradium is out of stealth to solve voice. We raised $70M and after only 3 months we’re releasing our transcription and synthesis products to power the next generation of voice AI.

1/2 We’re releasing an in-depth tutorial on neural audio codecs, the secret sauce that makes it possible for audio LLMs to not sound like a horror movie:

Rouard Simon, Orsini Manu, Roebel Axel, Zeghidour Neil, D\'efossez Alexandre, "Continuous Audio Language Models," arxiv.org/abs/2509.06926

Kyutai Speech-To-Text is now open-source! It’s streaming, supports batched inference, and runs blazingly fast: perfect for interactive applications. Check out the details here: kyutai.org/next/stt

Talk to unmute.sh 🔊, the most modular voice AI around. Empower any text LLM with voice, instantly, by wrapping it with our new speech-to-text and text-to-speech. Any personality, any voice. Interruptible, smart turn-taking. We’ll open-source everything within the next few weeks.

Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧. Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the source speech. Based on objective and human evaluations, Hibiki outperforms previous systems for quality, naturalness and speaker similarity and approaches human interpreters. 🧵

``MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling,'' Simon Rouard, Robin San Roman, Yossi Adi, Axel Roebel, ift.tt/Ldh3kU1

Meet Helium-1 preview, our 2B multi-lingual LLM, targeting edge and mobile devices, released under a CC-BY license. Start building with it today! huggingface.co/kyutai/helium-…

I am presenting our paper MusicGen-Style “Audio Conditioning for Music Generation via Discrete Bottleneck Features” at @ISMIRConf this afternoon. The code as well as the weights of the model are available on github.com/facebookresear…. You can now play with it!

It start now at poster 227!

The code and weights of the model will be released soon. Stay tuned!

Then we can as well use text and style conditioning to generate music, but we noticed that the model tends to ignore the text prompt. We then introduce a double classifier free guidance. This guidance could be applied to other multi-conditioned generative models.

Very happy to announce that my paper “Audio Conditioning for Music Generation via Discrete Bottleneck Features“ done with @honualx @adiyossLC @jadecopet and Axel Roebel has been accepted at ISMIR24. Paper: arxiv.org/abs/2407.12563 Sample: musicgenstyle.github.io Code: soon

``Audio Conditioning for Music Generation via Discrete Bottleneck Features,'' Simon Rouard, Yossi Adi, Jade Copet, Axel Roebel, Alexandre D\'efossez, ift.tt/0DRZW2F

#ICML2024 paper “An Independence-promoting Loss for Music Generation with Language Models” We promote independence between EnCodec codebooks using a kernel trick and improve music generation quality 🎶 Paper 📜 arxiv.org/pdf/2406.02315 Audio/Code 🔊 jmlemercier.github.io/encodec-mmd.gi…

``An Independence-promoting Loss for Music Generation with Language Models,'' Jean-Marie Lemercier, Simon Rouard, Jade Copet, Yossi Adi, Alexandre D\'effosez, ift.tt/eDpVoCf