NovaSky
153 posts
@NovaSkyAI
Building SkyRL at @BerkeleySky Join the Slack community: https://t.co/mSO97T61vR
SkyRL now implements the Tinker API. Now, training scripts written for Tinker can run on your own GPUs with zero code changes using SkyRL's FSDP2, Megatron, and vLLM backends. Blog: novasky-ai.notion.site/skyrl-tinker 🧵
Today, we're excited to share that Biomni is published in @ScienceMagazine. Biomedical research is still fragmented, manual, and difficult to scale. In this work, we introduce Biomni - the first general-purpose biomedical AI agent with an integrated biology environment that can reason, plan, and execute end-to-end scientific workflows. We show that, with the right environment and harness, AI can automate large-scale omics analyses, orchestrate laboratory robotics, optimize molecular properties, and even train new AI models for biology. We also introduce a reinforcement learning recipe for continually improving biomedical AI agents, enabling open-source models to achieve frontier-level performance. It's surreal to look back. We started the Biomni project in early 2024, when agentic AI was still nascent. It is exciting to see tens of thousands of biologists collaborating with agents every day to accelerate science. Try Biomni: biomni.phylo.bio Read more: science.org/doi/10.1126/sc… This work is not possible without this truly inter-disciplinary team: @serena2z @hcwww_ @YuanhaoQ Minta Lu, Ryan Li, @yusufroohani Lin Qiu @shiyi_c98 Gavin Junze Di @rickwierenga @kavi_deniz Sherry @TianweiShe Shruti Jennefer Xin Zhou @MWheelerMD Jon Bernstein @MengdiWang10 @PengHeAtlas @zhou_jingtian @SnyderShot @lecong Aviv Regev @jure @StanfordAILab @genentech @phylo_bio @arcinstitute @UW @berkeley_ai @RetroBio_ @tamarindbio @Princeton @UCSF
Excited to share some of our work on improving vLLM for RL! A number of RL frameworks, including SkyRL, use vLLM for inference, and we’ve noticed some common problems: 1. Weight syncing between training and inference is implemented in an ad-hoc fashion and duplicated across frameworks. 2. Asynchronous RL is prone to break at scale, especially in P/D and DPEP deployments. We’ve been working on improving both!
Today, @MichaelElabd, @QuantumArjun, and I are excited to announce Trajectory. We are a research lab and product company building the platform for Continual Learning. Our platform unlocks the signal already sitting in product usage, so companies can continuously post-train large-scale agentic models that outperform the frontier. @trajectorylabs We’ve raised $15M from @Conviction, @BessemerVP, @radicalvcfund, @jeffdean, @drfeifei and more. We’re partnering with some of the best AI-native companies: @ClayRunHQ @Harvey, @DecagonAI, @mercor_ai, @RogoAI to power their agentic systems, some of which we are already in production with. We’ve brought together a world class research team from DeepMind, OpenAI, Apple, Meta Superintelligence, Amazon AGI, Scale AI, and an elite product team from Stripe and Figma. AI will never again start on day one. Every correction, every retry, every edit will make products smarter. This is Continual Learning.
So excited to share that I’ve joined @trajectorylabs! We’re pushing the frontier of RL research to build the platform for continual learning - systems that learn and evolve alongside your products in real time. We believe in a world where everyone has the power to own their own intelligence and shape their own destiny. And we’re hiring :)
Reinforcing Recursive Language Models Can a 4B model learn to recursively call itself to answer hard long-context questions? We RL fine-tuned a small model to behave as a native RLM. On evidence selection across scientific papers, our 4B RLM matches Sonnet 4.6 in quality while running significantly faster and cheaper.
SkyRL now supports end-to-end vision-language post-training, from SFT to agentic RL, and adds vision model support to SkyRL’s Tinker interface! Existing multimodal cookbooks, e.g. VLM classification, work out of the box:
Can we train code agents to search relevant locations in a codebase only using a terminal? Introducing CodeScout: an effective RL recipe for code search 🚀 🏆 Outperforms 18x larger OSS LLMs 🔥 Comparable to proprietary LLMs 📈 SoTA on SWE-Bench Verified, Pro, & Lite 🧵 [1/N]
Hi all, extending the invite to the LLMs on Ray office hours next Thursday, 3/5 9:30-10:30AM PT! We will be hosting @erictang000 and @sumanthrh from the @NovaSkyAI SkyRL project to present on inference/vLLM in RL and take questions from the group. After, there will be time for any other questions folks have on distributed inference w/ Ray. Hope you can make it! Sign up for the invite here: forms.gle/QESMQ8ojRJsCZV…
Introducing our new work K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model — a new paradigm for automated GPU kernel generation, achieving SoTA results. 🔍 Big insight: Traditional methods treat LLMs as stochastic code generators inside heuristic loops — but this misses a key point: LLMs are powerful planners with rich domain priors. 🧠 Core idea: K-Search uses the LLM itself as a co-evolving world model — one that plans + updates beliefs + guides search decisions based on experience. 📌 This decouples high-level strategy (intent) from low-level code implementation, allowing the optimizer to pursue multi-step transformations even when intermediate implementations don’t immediately improve performance. 📈 Key results: 🔥 Our discovered kernels are ~2.10× average speedup vs state-of-the-art evolutionary search across 4 FlashInfer kernels on H100/B200. 🔥 Up to 14.3× gain on complex Mixture-of-Experts (MoE) kernels. 🔥 State-of-the-art performance on GPUMode TriMul (H100) task — beating both automated and human solutions. 🙏 Acknowledgements This work is developed in @BerkeleySky, w/ the amazing @ziming_mao, @profjoeyg, and @istoica05. We thank @DachengLi177, @MayankMish98, @randwalk0, @pgasawa, @fangz_zzu, and @tian_xia_ for helpful discussion and feedback. We also thank the generous compute support from @databricks, @awscloud, @anyscalecompute, @nvidia, @Google, @LambdaAPI, and @MayfieldFund. 👨💻 GitHub: github.com/caoshiyi/K-Sea… 📄 arXiv: arxiv.org/pdf/2602.19128…
