Lorenzo
131 posts
it’s surprising to me how many people seem to not understand that great models are built with super high quality curated data
finding novel ways to create / get this data is a huge edge
English
made a tiny app so you can try to beat vlms on the task autoresearch comes up with
Lorenzo@lsteno
we launched an autoresearch run to find interesting questions for vlms and they're so cute
English
we’re hosting an event in zurich about curiosity in agents, check it out!!
Seldon@seldon_tech
Join us for our first event on July 19th, a talk about Auto-Research Agents with @matteosaponati, curator of Tufa Talks. We will explore the current state of auto-research and talk about if novel architectures are needed going forward- Spots are limited, sign up link below
English
as a physicist this reminds me little bit of smth
OpenAI@OpenAI
Introducing a limited preview of GPT-5.6 Sol, our next generation frontier model, as well as GPT-5.6 Terra, a balanced model for efficient, everyday work, and GPT-5.6 Luna, a fast and affordable model for high-volume work. openai.com/index/previewi…
English
Lorenzo retweetledi
New blogpost: What does your benchmark actually measure?
I argue what evals end up measuring is the simplest way to improve performance on them. When building a new benchmark, we are often too focused on minimizing scores at launch.
This often makes benchmarks too easy to increase scores on, without reflect improvements in the underlying capability it intended to measure.
Instead, I find it useful to think about, and apply, all the ways someone could improve scores on a benchmark. This often reveals unintended shortcuts that were enough to make large eval gains.
Thinking about how a benchmark behaves under optimization pressure helps interpret results on it, and predict behaviors that will surface in future runs, as if it's successful, the community/models will try everything to hill-climb it.
Full post below:
English
Lorenzo retweetledi
