Matteo Gay

The Sleep of Reason Produces Slop

New paper on a long-shot I've been obsessed with for a year: How much are AI reasoning gains confounded by expanding the training corpus 10000x? How much LLM performance is down to "local" generalisation (pattern-matching to hard-to-detect semantically equivalent training data)?

LLMs are injective and invertible. In our new paper, we show that different prompts always map to different embeddings, and this property can be used to recover input tokens from individual embeddings in latent space. (1/6)

A feast of SAC Highlights awards at @aclmeeting for my students! Lead authors: - Nina Gregorio & @gaiusmets for "The Cross-linguistic Role of Animacy in Grammar Structures" - @GiwonHong413849 for "Mixtures of In-Context Learners" Massive congrats! 2025.aclweb.org/program/awards/

One of my main motivations for founding Our World in Data was to overcome the mad reality in academia that much of the most important research and data is locked away behind paywalls. I wanted research and data to be a public good, accessible to *everyone*. If you find our work valuable, I would appreciate if you would pay us a little so that we can make good data and research accessible to everyone. ourworldindata.org/donate

We just released Mixtral 8x22B. Super excited for this release!

@conrad_anker and @leo_houlding dressed as George Mallory and Sandy Irvine on Everest, back in 2007. The question of whether Mallory and Irvine may have made it to the summit of Everest before disappearing in 1924 has remained one of climbing's most enduring mysteries.

Modeling the interaction between tokens is vital to NLP. It takes quadratic time—every pair of tokens needs to be compared. Our #EMNLP2023 paper shows that, by casting this problem as sorting, this complexity can be reduced to linear—just like you can sort n numbers in O(n).

But there's more: a few days ago, @Youssef_M_Nader, working with new segmentations from the Vesuvius Challenge segmentation team, was able to generate this image of stunning clarity and size: This is the revolution in digital papyrology that Dr. Seales has created. Nothing like this has ever happened before and papyrology will never be the same. The last couple of months have been intensely electric for those of us working on the Vesuvius Challenge, and if progress continues, the coming months promise a deluge of new ancient text.

@AntigoneJournal Hesiod, Works and Days & J.-P. Vernant, The Origin of Greek Thought

A question for our readers: what is the first book you read about the Ancient Greeks, Romans, or Ancient Mediterranean world that made you think, "This is fascinating and I want to learn more?" (We are thinking of gathering a collection of inspirational first books, old and new.)

The debate over AI capabilities often hinges on testing abilities that humans are presumed to have — reasoning, logic, systematic, compositional generalization, grammar, etc. But how reliably good are humans actually at these skills? 1/

I saw the best minds of my generation fail to connect to eduroam

Stefano Ghisolfi makes FA of Excalibur (F9b+) Ghisolfi has kicked off his 2023 season in style making the first ascent of Excalibur, his long-term project at Drena, Arco. climber.co.uk/news/stefano-g…

Aidan Roberts has repeated Alphane (Font 9A) at Chironico, in Ticino, Switzerland. This is the hardest a British climber has bouldered! ukclimbing.com/news/2022/10/a…

I think a lot of people that I know personally would fail the Turing test