Isaac Tamblyn

So, this is it then? No more OOMs of scaling, no more speculation about whether GPT-5, 6 or 7 would be AGI. The performance wasn't even good enough to brand it as GPT-5. The diminishing returns finally kicked in and this is as far as we'll get just from scaling pretraining.

Hopefully KC sobers up by the 2nd half

Anyone else having audio issues with @TSN_Sports right now? It’s all static #SuperBowlLIX

ChatGPT model naming conventions are terrible. The interface has no less than 7 different model options to choose from...

Pictured: me giving up on language model research in 2011.

🤣🤣

Headlines about inflation are getting annoying. Why do people expect prices to go DOWN, when inflation is still a positive number? cbc.ca/news/business/…

How do you make people ignore a $60 billion dollar hole? Fire someone famous.

I just assumed Trudeau was sticking around until he could go to the Eras Tour.

Congratulations to COP 475 for continuing to under deliver! Don't worry about it, I'm sure you will get it right next time!!! cbc.ca/news/world/cop…

LLMs are powerful sequence modeling tools! They not only can generate language, but also actions for playing video games, or numerical values for forecasting time series. Can we help LLMs better model these continuous "tokens"? Our answer: Fourier series! Let me explain… 🧵(1/n)

@olexandr @andrewwhite01 B3YLP is all you need

@andrewwhite01 I don’t complain 😉 We just generate our own data

Every AI chemistry talk ever: "We need more data!" never does anyone say "I'm so grateful for the data we have" or "here are 10 more data points." Just complaint after complaint from these chemists

Are you doing research in AI and looking for a full-time position in industry? Come work with me at Block. (we are a remote friendly company :) block.xyz/careers/jobs/4…

@FindingStephen Well done!!!

unpopular opinion: ML conferences should charge $100 per submission. For accepted papers this would count towards the registration fee of the attending author, so it's free. Extra funds collected could be used eg. for replication studies or other improvement to the review process

@TD_Canada your implementation of interac is broken. If you hold a transfer for risk / fraud, you need to tell the customer, otherwise they wait around not understanding what happened.

Let me share a lecture from my ML for Physics course at @EPFL_en, where I derive the Boltzmann machine from the maximum entropy principle and use it to motivate the attention mechanism and a simple transformer architecture. Enjoy! youtu.be/dwMhw2X8_TU youtu.be/od_XiaCJzV0

This mini-review is based on Hugo Cui's PhD thesis: arxiv.org/abs/2409.13904 . My advice to him was: "Write something you would have loved to have when you started your PhD!" He did an outstanding job introducing the rich methods he developed. Enjoy and share widely!

Today the Nobel committee has confirmed my long-held definition of Physics: Things I find interesting.

John Hopfield has a nice article in the annual reviews of condensed matter physics. It starts off with a discussion of what physics is, which I think is totally on point.