Gavin Uberti

@tanishqkumar07 @tri_dao @avnermay Super cool stuff!

I've been working on a new LLM inference algorithm. It's called Speculative Speculative Decoding (SSD) and it's up to 2x faster than the strongest inference engines in the world. Collab w/ @tri_dao @avnermay. Details in thread.

Ben and Asher are some of the smartest people I know, and the team they've assembled is world class as well. So excited to see what they build!

@andrei_serban @ThriveCapital @scale_AI @tryramp @webflow Exciting stuff!

All companies run on IT. But almost all IT teams are underwater. The ones that aren't run on Console. Console raised a $23M Series A led by DST Global Partners and @ThriveCapital to help leading companies like @scale_ai, @tryramp, and @webflow automate 50%+ of their tickets.

Mercor (@mercor_ai) scaled from $1-500M in revenue run rate in the last 17 months, making us the fastest growing company of all time. Our growth is accelerating. We averaged 11% week over week growth in July, 18% WoW growth in August, and 19% WoW growth in September. One trend driving this meteoric growth: the Economy is Becoming an RL Environment Machine. Reinforcement learning is becoming so effective that agents can hillclimb any benchmark, but humans need to define the rewards to automate everything. While everyone fears job loss, we’re creating a new category of knowledge work faster than any other time in history. The future of work will converge on training agents. We're paying out over $1M / day to people in our marketplace and hiring experts rapidly across nearly every domain: software engineers, doctors, lawyers, consultants, bankers, and many more.

@TarunAmasa @OpenAI Congrats Tarun! Exciting stuff!

It’s official. We’ve raised $14m led by @OpenAI Startup Fund to bring AI to Excel. Endex is the first AI agent to live inside Excel. For the past year, we've been working with financial firms. Today we’re releasing it to the world. Our capacity is limited; comment below for an early invite 🧵

There is no data wall.

happy llama day to all those who celebrate

@JoeLi5050 @radi_cho @zeynebnkaya @nicolesplaining Congratulations guys! Well deserved

Just won 1st Place and $40k at the Mercor x Etched x Cognition inference-time compute hackathon! 4 Stanford freshman (@radi_cho, @zeynebnkaya, @nicolesplaining, and I) built "LLaDA-R1: Scaling Reasoning at Inference Time with Diffusion-LLMs" in 24 hours

@magnificentgrnt The trustees behind Magnificent are some of the best mentors I've had the privilege of working with. Salar, Firat, Sidar, and of course Barend are all world-class.

Check out the new 2024 Cohort of Magnificent Grants. And happy to report that going forward, the fellowship application process will be on a rolling basis, launching a nomination system... substack.com/home/post/p-15…

Excited to be part of the founding class of Magnificent Grants, and congratulations to all the winners!

@vllm_project Would the additional reasoning tokens allowed by the speedup make up for the loss in accuracy (assuming we are in a time-bound environment)? Come test it at the Inference Time Compute Hackathon!

@vllm_project Rather than insist the speculative decoding distribution match exactly, if you are OK with, say, 99% of the distribution being recovered (e.g. according to some metric like KL Divergence), you could accept guessed tokens more frequently

Hackathon idea - nearly speculative decoding. As of v0.7.3, @vllm_project supports Deepseek R1's Multi-Token Prediction module, letting you "skip" a token generation if the multi-token prediction guessed it correctly in advance. But what if you accepted almost correct guesses?

@akashorion @akashorion come test it out!

You could add more experts by just increasing K_r and M. Would this make Deepseek smarter? Would you need to scale down the outputs accordingly? Would you need to fine-tune a bit, or would it work out of the box? What if you used all experts simultaneously?

Using more FLOPs should make transformers smarter. DeepSeek R1 currently uses ~8 routed experts per token. So would selecting more (and possibly scaling by the router) improve performance? Come test it out at the Inference Time Compute Hackathon and win up to $60k in prizes!

For intelligence, FLOPs are all you need. So I'm excited to announce our Inference Time Compute Hackathon with @cognition_labs, @mercor_ai, @CoreWeave, and @AnthropicAI. When exaFLOPs are too cheap to meter, what will we build?

@juberti @OpenAI Congratulations to Sam for nabbing a world-class engineer!

During the development of WebRTC, we recognized the impact of voice and video on human communication, and I wondered if someday we'd talk to AIs the same way. Today, we can see this future taking shape, and I'm excited to announce I've joined @OpenAI to lead real-time AI efforts!

@Tanishq97836660 Very cool!

Our preprint is on arXiv at [arxiv.org/pdf/2411.03541]

[1/5] Made a Twitter account just to share some fun research projects I worked on over the summer! The first is a computational neuroscience project asking: do mice grok? TLDR; we revisit recent neural data from mouse cortex as they are overtrained on a binary odor discrimination task, finding that they continue to learn generalizing solutions -- their odor representations of the two classes continue to separate -- even as behavior stops changing on a training set. Joint work with @blake__bordelon, @CPehlevan, Venki Murthy at @MCB_Harvard and @gershbrain!