Akash Srivastava

come check out poster #5518 at NeurIPS morning session today to learn about how you can encourage diversity / prevent early-pruning during inference-time scaling and boost the performance of any model without additional training!

@annadgoldie @Azaliamirh @RicursiveAI Congrats @annadgoldie and @Azaliamirh — very exciting!

Excited to announce that @Azaliamirh and I are launching @RicursiveAI, a frontier AI lab creating a recursive self-improving loop between AI and the hardware that fuels it. Today, chip design takes 2-3 years and requires thousands of human experts. We will reduce that to weeks. This will be incredibly hard. For context: Go has a search space of 10^360. A simplified version of chip placement—only one part of the design process—has a search space of 10^9000. But we are the right team to solve it. We co-founded the Machine Learning for Systems team at Google Brain. There, we built AlphaChip—an RL agent for chip placement. AlphaChip has been used to design four generations of TPUs, data center CPUs, autonomous vehicle chips, and mobile phone chips. These chips are running in data centers and devices all over the world. Our immediate goal is to dramatically accelerate chip design. Next, we plan to design chips end-to-end given an ML workload, unlocking a Cambrian explosion of custom silicon. Finally, we will close the recursive loop. We will build our own chips, train our own models, and co-evolve them on the path to superintelligence. AI designs better chips 🔄chips train better AI We sat down with @WSJ’s @Berber_Jin1 to discuss Ricursive: wsj.com/tech/this-ai-s…

What does it take to scale AI beyond the lab? At #RedHatSummit, @ishapuri101 and I spoke with Red Hat CEO Matt Hicks & CTO Chris Wright on inference-time scaling, open infra (LLMD), and making AI affordable for enterprise. 🎧 youtu.be/mj1dwrPfvb4 #NoMathAI @RedHat_AI

🚀 How is generative AI transforming the way we design cars, planes, and entire systems? In Ep 2 of No Math AI, @ishapuri101 and I chat with Dr. @_faezahmed (@MIT DeCoDE Lab) about how AI boosts creativity, cuts design time, and works with engineers—not against them.

SQuat: KV-Cache for making reasoning models go 🚀 📄paper: lnkd.in/emKhAVZu 💻 code: lnkd.in/e8TJ7N3R From my awesome collaborators @RedHat_AI

Excited to share our preliminary work on customizing reasoning models using Red Hat AI Innovation’s Synthetic Data Generation (SDG) package! 📄 Turn your documents into training data for LLMs. 🧵👇

had a great time giving a talk about probabilistic inference scaling and the power of small models at the IBM Research ML Seminar Series - the best talks end with tons of questions, and it was great to see everyone so engaged : ) youtube.com/watch?v=--3rsQ…

Come along and help us build reasoning in small LLMs

Excited to share our latest work with @ishapuri101 et al.! 🚀 We introduce a probabilistic inference approach for inference-time scaling of LLMs using particle-based Monte Carlo methods—achieving 4–16x better scaling on math reasoning tasks and O1-level performance on MATH500.

🧩 Why do task vectors exist in pretrained LLMs? Our new research uncovers how transformers form internal abstractions and the mechanisms behind in-context learning(ICL).

Neural activity is correlated among animals performing the same task and across sequential trials. Led by @zhang_yizi and @hl3616, we develop an reduced-rank model that exploits shared structure across animals to improve neural decoding. biorxiv.org/content/10.110…

What will a foundation model for the brain look like? We argue that it must be able to solve a diverse set of tasks across multiple brain regions and animals. Check out our preprint where we introduce a multi-region, multi-animal, multi-task model (MtM): arxiv.org/abs/2407.14668

🚀 Stronger, simpler, and better! 🚀 Introducing Value Augmented Sampling (VAS) - our new algorithm for LLM alignment and personalization that outperforms existing methods!

Excited to give a talk on our hottest, newest work “Value Augmented Sampling for Language Model Alignment and Personalization” at 2:30p Halle A3 in #ICLR2024 Reliable and Responsible Foundation Models Workshop 🥳🥳

Attending #ICLR2024, interested in continual learning and like probabilistic modeling? Lazar from the @MITIBMLab, will be presenting our latest work that takes a probabilistic approach to modular continual learning on Tuesday, 7 May, Halle B #222 (iclr.cc/virtual/2024/p…).

Check out our work titled "From Automation to Augmentation: Redefining Engineering Design and Manufacturing in the Age of NextGen-AI", where we highlight the requirements for NextGenAI suitable for design, engineering, and manufacturing. mit-genai.pubpub.org/pub/9s6690gd/r…

@jeethu @Geronimo_AI Good point, model card has been updated with v2 results.

@Geronimo_AI They seem to be comparing it against the older Mistral-7B-Instruct-v0.1 and not the newer Mistral-7B-Instruct-v0.2 model.

New work from @MITIBMLab researchers on large scale alignment of LLMs. Check out the models at HF huggingface.co/ibm/merlinite-…

New work on automated red-teaming in LLMs using curiosity-driven exploration! #iclr24