Aurko Roy
107 posts
@aurko79
Math & computer science | @AIatMeta (2025-2025) | @GoogleDeepmind (2023-2025) | @GoogleAI (Brain) (2017-2023) | CS PhD @Georgiatech | CS @IITKanpur
Michael Rabin passed away. Sic transit :-( en.wikipedia.org/wiki/Michael_O…
🚀 Scaling embeddings, not just experts—introducing a new path for efficient LLMs. Key Finding: In high-sparsity scenarios, N-gram embeddings yield a better Pareto frontier than just adding more MoE experts. Therefore, we introduce LongCat-Flash-Lite—the first opensource model built on this insight. ⚙️ 68.5B Total Params(37.13B non-embedding) | 2.9B~4.5B Active 📊 High Performance: SWE-Bench 54.4 | τ²-Bench 72.8 | TerminalBench 33.75 📃 256K Context Window (YARN-powered) ✨ Optimized for Agentic/Coding, strong in general reasoning ⚡ ~700 tokens/s peak inference speed The result: Achieves competitive performance within its scale at a significantly lower cost and latency. Hugging Face: huggingface.co/meituan-longca… Tech Report: huggingface.co/meituan-longca…
We are thrilled to announce a strategic partnership with Google! Google is also making a financial investment in Sakana AI to strengthen this collaboration. This underscores their recognition of our technical depth and our mission to advance AI in Japan. We are combining Google’s world-class products with our agile R&D to tackle complex challenges. By leveraging models like Gemini and Gemma, we will accelerate our breakthroughs in automated scientific discovery. Our work on The AI Scientist and ALE-Agent has already demonstrated the power of these models. Now we are going further. We are scaling our deployment of reliable AI in mission-critical sectors. We are working with financial institutions and government organizations to deliver solutions that meet the highest standards of security and data sovereignty. We are excited to drive the widespread adoption of reliable AI and advance Japan’s AI ecosystem together!
In the docstring, Noam simply wrote: "Noam just made this up. Replacement for Zero++ gradient compression" and it ended up unblocking large scale run on almost impossible cluster topology.
Actually I was lucky to meet Grisha as a teen way before he became famous. I had an internship not far from a park where he used to walk, and he was going for a walk every day same route same time. He was quite famous in the neighborhood bc of his non standard look (according to slavic standards). His response to me on why he declined was that he didn’t feel like it was fair. He said he was building his proof on top of works of other great mathematicians and none of them got any medals, but without them he wouldn’t solved it
Today, we’re excited to introduce Rnj-1, @essential_ai's first open model; a world-class 8B base + instruct pair, built with scientific rigor, intentional design, and a belief that the advancement and equitable distribution of AI depend on building in the open. We bring American open-source at par with the best in the world.
We are beyond thrilled to share our first flagship models, Rnj-1 base and instruct 8B parameter models. Rnj-1 is the culmination of 10 months of hard work by a phenomenal team, dedicated to advancing American SOTA OSS AI. Lots of wins with Rnj-1. 1. SWE bench performance close to GPT 4o. 2. Tool use outperforming all comparable open source models. 3. Mathematical reasoning (AIME’25) nearly at par with GPT OSS MoE 20B. ….
Introducing Claude Opus 4.5: the best model in the world for coding, agents, and computer use. Opus 4.5 is a step forward in what AI systems can do, and a preview of larger changes to how work gets done.
