Matt C.

Our models are way over parameterized, our learning algorithms are still incredibly slow, our machines are too power hungry, and our understanding of intelligence is too shallow. The path goes much, much further. It terrifies me how early we are

Theory is cheap; now onto real, practical problems. After using @OpenAI GPT 5.6 Sol for a while, it seems increasingly likely that LMs will reduce the need for empiricists to collaborate with theorists and increase the workload of verifying and demonstrating practical utility. In many ways, this is the time for those who understand theory and application, who can't beat theorists on theory but can't beat domain experts on application, to shine. Novel theories that used to take months/years to prove and inspire novel methods and applications can now be generated in a couple of hours -- set up 5.6 Sol with xhigh or ultra thinking and let it loose. The only thing that's left is to put together practical/real-life examples that verify the theory and beat baseline methods. The latter is much more difficult to do, but I'd imagine in a few months, this will change once LMs learn better taste. All of this really revolutionizes the research landscape.

Was there anything in the Medical Imaging space that people thought was really cool at #icml2026? Self-promotion welcome! Especially funky or creative ideas

Introducing the world's fastest tokenizer implementation, Gigatoken! Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines. These baselines are already multithreaded Rust implementations! 🧵

The Codex cybersecurity risk is more of a nuisance than a help to the broader community tbh. I'm running an ML research workflow on agents and distribution shift, and it triggered when using the publicly available network intrusion dataset, which included what it claimed was "cybersecurity risk" language. However, when prompting Codex for a justification, it told me it was definitely not a risk 🤦‍♂️. Not sure what the best solution is for this, since the current filter seems to be keywords. This needs to be updated asap @OpenAI . "Your request is not meaningfully a cybersecurity request. It is an ML research workflow on agents and distribution shift. The likely trigger was my use of CIC-IDS2018, a network-intrusion dataset. Automated filters may react to words such as “attack,” “injection,” “infiltration,” or “harmful actions,” even though here they refer to: evaluating a labeled academic dataset; measuring statistical drift; testing prompt-injection defenses in AgentDojo; detecting unsafe agent behavior in sandboxed environments. None of this involves compromising systems, deploying malware, stealing credentials, or providing offensive instructions. The work is defensive and experimental, so there is no substantive reason the research content itself should be withheld. If something was hidden, it was most likely an overly broad automated classification rather than a limitation inherent to your request." The workaround is not to use the dataset 🤷‍♂️.

Anthropic's main competitor in 5 years is going to be Genentech

Interesting technique still widely used in frontier models: activation checkpointing, where, in the forward pass, we keep only activations from a subset of layers, and, in the backward pass, we recompute the missing activations from the last checkpoint. Thus, trading memory for compute. In terms of computational complexity, checkpointing all layers is O(1), where L is the number of activation computations. Checkpointing no layers/recomputing activations is O(L^2). Nowadays, it seems activation checkpointing is determined at the hardware level. For example: - Checkpoint whole Transformer blocks when memory is tight. - Checkpoint only expensive-memory, relatively cheap-compute submodules, such as attention internals. - Combine checkpointing with tensor, pipeline, sequence, and data parallelism - Tune the policy for sequence length, microbatch size, model width etc NVIDIA's current Megatron stack supports full-layer and selective activation recomputation to target memory-intensive operations that are cheaper to recompute. Full transformer-layer recomputation can add roughly 30% compute overhead. docs.nvidia.com/nemo/megatron-…

One thing that blew my mind (that might be obvious to some) when learning about resource allocation from Percy's LM from Scratch is that once we do the math, training is compute-bound, but inference is memory-bound. So AI companies have lots of GPUs for training, which do many matrix multiplications. But this isn't the main bottleneck for consumer products. For a regular consumer, we prompt models (aka do inference). These are instead matrix-vector multiplications, NOT matrix multiplications. Matrix multiplication is compute-bound, while matrix-vector multiplication is memory-bound. This is the main reason why memory companies (micron, samsung, sandisk, western digital etc.) are in such high demand and I don't expect them to stagnate any time soon. Here are some back-of-the-envelope calculations (assuming bf16=2 bytes). The measure is arithmetic intensity=FLOPS/bytes: 1. Dot product x@w=y: - Bytes (read x, read w, write y)=2n+2n+2=4n+2 - FLOPS=n multiplications + (n-1) additions = 2n-1 - Arithmetic Intensity=(2n-1)/(4n+2)~1/2 2. Matrix vector product w@X=y where w is nx1 and X is nxn: - Bytes (read w, read X, write y) = 2n+2n^2+2n=2n^2+4n=2n(n+2) - FLOPS=n(2n-1) - Arithmetic Intensity=FLOPS/bytes=n(2n-1)/(2n(n+2))~1 - During inference, matrix vector product is what happens, so inference is memory bound! 3. Matrix multiplication X@Y=Z where X and Y are nxn - Bytes (read X, read Y, write Z)=n^2+n^2+n^2=3n^2 - FLOPS=n^2(2n-1) - Arithmetic Intensity=FLOPS/bytes=n^2(2n-1)/3n^2=(2n-1)/3~n/3 - Arithmetic Intensity scales with matrix size! Takeaways: - Arithmetic Intensity scales by n in matrix multiplication (training: compute-bound), but is constant for matrix vector products (inference: memory-bound). - Intuition: sending n^2 bytes to accelerator and getting n^3 bytes back - This is the reason for large batch sizes and large matrix multiplications in transformers

Best World Cup half time show ever! Madonna, BTS, Justin Bieber, Shakira. Might be even better than the game itself.

Some thoughts on the evolution of language models: 2018 (BERT): something you fine-tune aclanthology.org/N19-1423/ 2020 (GPT-3): something you prompt arxiv.org/abs/2005.14165 2022 (@OpenAI ChatGPT): something you talk to 2026 1st half (Agents): something that acts autonomously 2026 2nd half (@thinkymachines Inkling): something that interacts with the world thinkingmachines.ai/inkling/ Near future 2026+: I think we will see widespread use of agents that learn and experience the world on long horizons (possibly multimodal later). We're already seeing long-horizon benchmarks and progress (arxiv.org/abs/2607.08964… arxiv.org/abs/2607.09175 arxiv.org/abs/2607.09322). Not to mention rapid progress in world models and simulation, agentic continual/lifelong learning, and embodied AI. I can't help but think back to Richard Sutton's Bitter Lesson essay incompleteideas.net/IncIdeas/Bitte… . Maybe Sutton was right after all...

@HuskyAiii After 5.6 Sol came out, it’s not comparable now. GPT usage limits and quality are much better than Claude, especially when using the supposedly cheaper Opus 4.8 — Fable 5 is too expensive. And the GPT subscription comes with a more comprehensive suite of tools (image gen etc.).

What do you think? Is Codex is wayyy better (now) than Claude code

@yuvalmarton Thanks for the suggestions! Fortunately my initial reviews turned out fine, but it would be much more demoralizing for the less fortunate. I’m affiliated with a SAC and will bring this up. Hopefully we’ll see changes soon. ARR doesn’t disappoint with speedy policy changes.

@matte_ce I hope #ACL / #ARR will make the proposal above mandatory and part of the reviewing guidelines going forward. #EMNLP2026 #NAACL2027

Related: to avoid the frustrating “thx for this, I will keep my score”, I suggest we ask reviewers to state: if authors can explain / clarify / provide analysis of this, I will raise my soundness score to S and overall score to V. (Where S,V >= 4 ideally) #ACL #ARR #EMNLP2026

@iScienceLuvr @DrellLabs ML conferences are putting us several AI years behind

@DrellLabs I was at ICML

wait when did this happen?

@bakkermichiel Well I guess the story is wrong. Most likely they have the compute. Based on various reports, they’re moving to in house or domestic GPU alternatives. It’s going to be interesting how all of this unfolds.

Can someone explain me how the new Kimi can be so insanely good? The story has been "chinese labs have much less compute but they still stay close to the frontier through distillation". These results seem impossible to explain through distillation alone.