Richard Saunders

Our multi-model playground is coming together. Give it a try: nat.dev

OpenAI is creating an entirely new class of generative models for images.. arxiv.org/abs/2303.01469

🐱 Attention everyone! 🐱 🙀 Big news from OpenAI! 🙀 🤖 The API Playground now has a Chat mode! 🤖 😻 thanks to @nadipity @_chestercho 😻

AI agents can now teach themselves HOW to use tools (ie. any API) in real time, completely automated! Introducing: Self-Learning Agent for Performing APIs (SLAPA) with @FinsamSamson

We built an interactive data frame powered by foundation models that can wrangle your unstructured data (images, videos, text docs...) Introducing 🔮 Meerkat! 📃 hazyresearch.stanford.edu/blog/2023-03-0… 💻 github.com/HazyResearch/m… 🌐 meerkat.wiki

I've been organizing an MLSys discussion group at MIT where we discuss influential/interesting ideas for building machine learning systems. Our discussion is all over the place but I do my best to keep a record here: mlsys.ai.

Excited to announce our Deep Learning Tuning Playbook, a writeup of tips & tricks we employ when designing DL experiments. We use these techniques to deploy numerous large-scale model improvements and hope formalizing them helps the community do the same! github.com/google-researc…

Chain ⛓️ Rule(s) rules! Appreciation thread of one of the most interesting coincidences in machine learning. Two rules, both named "Chain Rule", happen to be absolutely critical to recent advances in ML & AI. A 🧵 on the Chain Rule of Probability & the Chain Rule of Calculus👇

MIT researchers found that massive neural nets (e.g. large language models) are capable of storing and simulating other neural networks inside their hidden layers, which enables LLM to adapt to a new task without external training: bit.ly/3jIiesP

Stanford CS 324 - Large Language Models - Lecture Notes 2022 A handy notes on various topics and techniques related to large language models. Notes are structured really well and there are pointers for recent/landmark papers in LLMs. stanford-cs324.github.io/winter2022/lec…

Very strong headwinds for any company producing low/mid-quality ML training data. LLMs now quickly & flexibly produce high-quality synthetic data for training: - Information retrieval ranking models - Instruction fine-tuned LLMs - CoT fine-tuned LLMs - RL on LLMs (RLAIF) - etc.

Yesterday I gave a lecture at @Stanford's CS25 class on Transformers! The lecture was on how “emergent abilities” are unlocked by scaling up language models. Emergence is one of the most exciting phenomena in large LMs… Slides: docs.google.com/presentation/d…

I gave a version of this talk "Scaling unlocks emergent abilities in language models" today at USC ISI. There is a video recording: youtu.be/Z_Qt737HG-0 Thanks Justin Cho @HJCH0 for inviting me and organizing!

I made a small chrome extension that uses LLMs to help generate spaced repetition prompts while reading.

🚜 Cannot believe it is almost 3 years since my 2020 post on variations of Transformer. I spent some time and did a big refactoring of that old post with new section structure and new papers. Still missing a few items tho, will add them in slowly: lilianweng.github.io/posts/2023-01-…

Can LLMs extract knowledge graphs from unstructured text? Introducing GraphGPT! Pass in any text (summary of a movie, passage from Wikipedia, etc.) to generate a visualization of entities and their relationships. A quick example:

Excited to share SingSong, a system which can generate instrumental accompaniments to pair with input vocals! 📄arxiv.org/abs/2301.12662 🔊g.co/magenta/singso… Work co-led by myself, @antoine_caillon, and @ada_rob as part of @GoogleMagenta and the broader MusicLM project 🧵

@eladgil 100% @ilyasut, he’s one of the clearest thinkers in the field.

I didn't find a great listing of Ethereum-related information dashboards, so I started cobbling one together. Please feel free to point me to something better, submit PRs to this one, or just tell me what's missing. github.com/superphiz/dash…

Polynomial commitment schemes are critical to Ethereum’s scaling solutions. These schemes will be used in Danksharding, as well as in the proof systems behind Scroll. What are polynomial commitment schemes? And how will they help scale Ethereum? scroll.io/blog/kzg