Gabriel Rojo

New work with @AlecRad and @DavidDuvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

Black-Scholes is wrong almost everywhere. And yet, it’s still the language of options markets. The reason: It’s the flat limit of a curved geometric pricing space. The volatility smile? That’s the curvature. Below we see where markets actually live in that space Preprint: papers.ssrn.com/sol3/papers.cf…

@TheStalwart Image generation (using GANs) already started in 2014 and researchers like Yann LeCun were talking about GANs in 2016 as something groundbreaking Probably the most difficult think to predict would have been the use of tools by LLMs (use of web search, run python code, etc)

If you had told an AI researcher in 2016 that in 6 or 7 years, computers would be easily able to communicate in language?Would it have been obvious to them that other non-obviously linguistic capabilities (image generation, web browsing, etc) would emerge around the same time?

I strongly suspect that Claude Mythos is a looped language model, as described in the paper "Scaling Latent Reasoning via Looped Language Models" from ByteDance The authors of that paper called out graph search as one of the areas where looping provides a huge theoretical advantage over standard RLVR. And look at where Mythos blows out its competitors the most

I made an interactive blog post about how JPEG image compression works: sophielwang.com/blog/jpeg

While waiting for DeepSeek V4 we got two very strong open-weight LLMs from India yesterday. There are two size flavors, Sarvam 30B and Sarvam 105B model (both reasoning models). Interestingly, the smaller 30B model uses “classic” Grouped Query Attention (GQA), whereas the larger 105B variant switched to DeepSeek-style Multi-Head Latent Attention (MLA). As I wrote about in my analyses before, both are popular attention variants to reduce KV cache size (the longer the context, the more you save compared to regular attention). MLA is more complicated to implement, but it can give you better modeling performance if we go by the ablation studies in the 2024 DeepSeek V2 paper (as far as I know, this is still the most recent apples-to-apples comparison). Speaking of modeling performance, the 105B model is on par with LLMs of similar size: gpt-oss 120B and Qwen3-Next (80B). Sarvam is better on some tasks and worse on others, but roughly the same on average. It’s not the strongest coder in SWE-Bench Verified terms, but it is surprisingly good at agentic reasoning and task completion (Tau2). It’s even better than Deepseek R1 0528. Considering the smaller Sarvam 30B, the perhaps most comparable model to the 30B model is Nemotron 3 Nano 30B, which is slightly ahead in coding per SWE-Bench Verified and agentic reasoning (Tau2) but slightly worse in some other aspects (Live Code Bench v6, BrowseComp). Unfortunately, Qwen3-30B-A3B is missing in the benchmarks, which is, as far as I know, is the most popular model of that size class. Interestingly, though, the Sarvam team compared their 30B model to Qwen3-30B-A3B on a computational performance analysis, where they found that Sarvam gets 20-40% more tokens/sec throughput compared to Qwen3 due to code and kernel optimizations. Anyways, one thing that is not captured by the benchmarks above is Sarvam’s good performance on Indian languages. According to a judge model, the Sarvam team found that their model is preferred 90% of the time compared to others when it comes to Indian texts. (Since they built and trained the tokenizer from scratch as well, Sarvam also comes with a 4 times higher token efficiency on Indian languages.

Social scientists working with materials requiring digitization can only study what machines can read. In practice, that means printed Latin-script documents from well-funded archives. In a new working paper, I show that Vision Language Models used zero-shot outperform every existing OCR system across every script evaluated, and I propose a pipeline for deploying them on new collections. I apply it to six archival collections spanning 1.8 million pages across six countries for under $1,900.

With Nano Banana 2, the potential for creativity extends beyond image generation and into usable, deployable software and tools. Here are some ideas for what you can build with it: — Travel apps grounded in live data: An app that indexes real train trips and visualizes weather data to generate a hyper-realistic view of what passengers on the train are seeing in real-time — UI prototyping: Utilizing the 512px resolution tier and native aspect ratios (like 8:1 for web banners or 1:8 for mobile sidebars), you can build a real-time UI brainstorming tool. For example, a designer could type "cyberpunk inventory menu background," and the tool would rapidly fire off dozens of variations with minimal latency — Conceptual art direction: With the new "High/Dynamic" thinking level, you can create a tool for film or game directors that handles complex prompts. The model will take the time to reason through intricate instructions (e.g., specific camera lenses, lighting setups, and character poses) to ensure strict adherence to the prompt before rendering the high-fidelity concept art Share what you've made in the replies below!

17,000 tokens per second!! Read that again! LLM is hard-wired directly into silicon. no HBM, no liquid cooling, just raw specialized hardware. 10x faster and 20x cheaper than a B200. the "waiting for the LLM to think" era is dead. Code generates at the speed of human thought. Transition from brute-force GPU clusters to actual AI appliances. taalas.com/the-path-to-ub…

Thrilled to announce a big upgrade to Gemini 3 Deep Think that hits new records on the most rigorous benchmarks in maths, science & reasoning - including 84.6% on ARC-AGI-2, 48.4% Humanity’s Last Exam without tools, and 3455 Elo rating on Codeforces!

Can AI agents design better memory mechanisms for themselves? Introducing Learning to Continually Learn via Meta-learning Memory Designs. A meta agent automatically designs memory mechanisms, including what info to store, how to retrieve it, and how to update it, enabling agentic systems to continually learn across diverse domains. Led by @yimingxiong_ with @shengranhu 🧵👇 1/

Video models serve as a good pretrained backbone for robot policies. Paper: arxiv.org/abs/2601.16163 Code: github.com/nvlabs/cosmos-…

@mariatsa51 @WorldScholar_ Nice Ottoman architecture

@WorldScholar_ kavala Greece 🇬🇷

We talk about the Colosseum. We talk about the Pantheon. But we don't talk enough about the Aqueduct of Segovia. It's one of the most impressive Roman structures standing today.

New NanoGPT Speedrun WR at 99.3s (-5.6s) with a bigram hash embedding that is added to the residual stream before every layer. Inspiration from Svenstrup et al 2017 paper on Hash Embeddings, and Deepseek's Engram. Modded-NanoGPT now uses fewer training tokens than its parameter count, a radical divergence from the 20x Chinchilla ratio. github.com/KellerJordan/m…

The math that made Wall Street billions pricing options just got ported to prediction markets This paper builds the first Black-Scholes equivalent for platforms like Polymarket Treating belief volatility as a quotable risk factor, with proper tools for hedging jump risk around elections and macro events. The paper is dense but worth it:

OpenAI Employee Alma Maters --- Source: linkedin. com/posts/josh-angle-816b3436_want-to-work-at-the-arguably-the-hottest-activity-7389688792737210368-6DOS/

New on the Anthropic Engineering Blog: Demystifying evals for AI agents. The capabilities that make agents useful also make them more difficult to evaluate. Here are evaluation strategies that have worked across real-world deployments. anthropic.com/engineering/de…

@PrettyCitiesX Beautiful Spain 🇪🇸

Cathedral Of Barcelona, Spain 🇪🇸

Who could have predicted (before, say, mid-2025) that Z.ai (Zhipu) would be the first LLM company to go public?

@q_u_i_m_ Absolutamente a nadie que gestione dinero o empresas a nivel internacional le interesa el PNV o CIU o cualquier otra invención que cuentas en tu tweet. A NADIE. Salid de la aldea.

@ggomezrojo Ja saben el que han de dir. Però si hagués dit “we are a basque company” tampoc hagués passat res, probablement molts dirigents dels USA o el propi Trump, ja ho coneixen: nabasque.eus/us_basque_popu…

És Josu Jon Imaz, expresident del PNV. Més enllà de la qüestió concreta, una de les coses que molts no saben del PNV és l’extraordinària xarxa de connexions internacionals que té (i força, als Estats Units). Per una banda tenen connexions polítiques i han situat gent seva a les institucions espanyoles i internacionals, i per l’altra també tenen connexions econòmiques a través dels peixos grossos que tenen en posicions clau, a l’economia estatal i internacional. A més a més, tenen també una mena de xarxa d’informació/intel·ligència, que els fa conèixer bé el terreny que trepitgen: busqueu informació a Google, us sorprendrà. Tot plegat ha fet del PNV una força amb molt de know-how en qüestions de poder, negociació, organització, influència, etc. Últimament (sembla) que ja no són el que eren, però sens dubte aquest és el tipus d’organització que més convé a una nació sense Estat. A Catalunya, l’última cosa que s’hi assemblava una mica era la CiU de Pujol, amb un evident excés de personalisme. Per això, després d’ell tot han estat amateurs a qui no despenjaven el telèfon.