Dan Lyth

@dscape Ah, sorry, fixed now

@danlyth wanna come speak in an ai conference about acoustic tokens?

@dscape Hey, DMs are open :)

@sesame @_apkumar youtu.be/bTcpNQH8ViQ?si…

We're a small team @sesame, but looking for great people to join. Check out this podcast with @_apkumar to get a feel for how we work. More info at sesame.com/team

@realmrfakename @sesame huggingface.co/sesame/csm-1b huggingface.co/spaces/sesame/… github.com/SesameAILabs/c… ❤️

@realmrfakename @sesame Yeah, we’re open-sourcing one of the base models (not fine-tuned with the voices you hear in the demo) in the next two weeks. Will be here: github.com/SesameAILabs/c…

We're just getting started at @sesame. Check out the demo here: sesame.com/voicedemo

@reach_vb @realmrfakename @sesame Hey, sounds good vb, will be in touch.

@danlyth @realmrfakename @sesame hey dan - vb from hugging face this side - would love to help w/ the release. I can't seem to access our shared slack anymore but my DMs are open! or vb[at]hf[dot]co

Kind words from @seanhollister 🙏. But we've still got a long way to go...

And we’re building hardware.

Nice overview of some of the things we've been working on @sesame. Always a pleasure working with @justLV.

Delighted to share a little glimpse of what we've been working on @sesame

@FluorescentGrey @stableaudio @StabilityAI @iScienceLuvr @jordiponsdotme @ednewtonrex @harmonai_org @zqevans @chrlaf

@stableaudio @StabilityAI cc: @iScienceLuvr @jordiponsdotme @ednewtonrex @harmonai_org @danlyth sorry for tagging all of you I just don't seem tobe getting anywhere with the support ticket system through the website

@stableaudio been trying to use stable audio and in the last 24 hours only 50% of my attempts to generate audio actually don't time out, is this a system wide issue or something with my account?

@Dorialexander @pleiasfr @huggingface This is awesome. Do you know approximately how many hours this comes to?

Big announcement: @pleiasfr releases a massive open corpus of 2 million Youtube videos in Creative Commons (CC-By) on @huggingface. Youtube-Commons features 30 billion words of audio transcriptions in multiple languages, and soon other modalities huggingface.co/datasets/PleIA…

Excellent work by @sanchitgandhi99 and @yoachlacombe reproducing the text-description-to-speech model I developed while at @StabilityAI 👏❤️

@HubertSiuzdak Congrats, sounds great! Any code/paper?

SNAC encodes audio into hierarchical tokens, similar to SoundStream, EnCodec, and DAC. It introduces a simple change: coarse tokens are sampled less frequently, covering a broader time span. It is designed mainly for language models to accurately capture long-form audio with a consistent structure

recently I've been experimenting with audio compression & vector quantization and I'm happy to present Multi-Scale Neural Audio Codec (SNAC), which can compress audio below 2 Kb/s with decent quality 🎧 Listen to the samples and see how it compares to the state of the art:

@erogol I was wondering the same thing, there’s not a lot of detail on that.

@danlyth one thing unclear to me how they expand BPE before decoding. be cause otherwise they need to learn to alight it with the output audio frames. do you see how?

Moving beyond naturalness and WER, they propose a set of sentences that test the model’s ability to deal with compound nouns, emotions, foreign words, paralinguistics (e.g. whispering if the text requires it) etc. etc. The full test set is included in the appendix. 👏 2/7

There are a bunch of other interesting elements to this work, and it’s worth a read. Plenty of examples on the demo site too. Nice work Mateusz Łajszczak, @guillecambara, Yang Li, and all the other contributors. arxiv.org/abs/2402.08093

The speech “de-tokenizer” (or decoder) is a convolutional model that’s streamable and 3x faster than their diffusion-based baseline (and also sounds better). It’s built around BigVGAN. 6/7