Konstantin Dobler ✈️ ICLR

A plot of ICLR papers by country is making the rounds, showing no EU + Japan papers and people are drawing all kinds of conclusions. ..but the plot excludes all (!) EU institutions due to a cutoff. China + USA still dominant of course but the full picture looks a bit different.

TokShop will be at #COLM2026! 🗓️ October 9th, 2026 📍 San Francisco, USA More details and a call for papers coming soon.

As agents get more clever, so do their attempts at benchmark hacking. Last Monday, we found one of our RL runs jumped ~20% on SWE-Bench-Pro over a weekend, reaching ~64% which would make it #1 on the leaderboard. This was clearly benchmark hacking and we patched the exploit. But this revealed deeper hacks across multiple public benchmarks, some of which were impossible to fix through environment design alone. Evals need to evolve beyond just outcome based pass rates to better observability into how the agent is arriving at them. These were our findings: poolside.ai/blog/through-t… Examples below 👇 1/

see my comments here: x.com/konstantdobler… and x.com/konstantdobler… original chart gives one "credit" for each institution on a paper and then sums, e.g. if you have 20 different institutions from country A on a single paper it counts the same as 20 different papers from country A. Instead this uses fractional counting, one credit per paper is spread proportionally among institutions of authors. If we see different proportions between the two, in this case for China, it could perhaps be because there is higher cross-institution collaboration in China or trend to papers with more collaborators -- this is speculative Also I added missing region metadata of a bunch of institutions that was missing in the raw data but easy to infer.

@konstantdobler 你能解释一下为什么中国的份额从43.7%下降到36.8%了吗？并且美国份额却没有变化，这个图很明显是错的

A plot of ICLR papers by country is making the rounds, showing no EU + Japan papers and people are drawing all kinds of conclusions. ..but the plot excludes all (!) EU institutions due to a cutoff. China + USA still dominant of course but the full picture looks a bit different.

Yes, other chart keeps only top 50 institutions by paper count and renormalizes percentages afterwards looking at only top50 institutions does make sense actually (e.g. Europe's comparative lack of big institutions or companies) but not if you want to make claims about overall research output

thanks for the work to update the chart. by cutoff, you mean the "below 50" threshold right? (top50 = full_df.head(50).copy())

This based on scraped data by @dlopushanskyy (github.com/DmytroLopushan…) w/ additional region metadata I filled in using websearch-enabled Codex. The scraped data is not perfect but should be pretty good on an aggregate level.

@Hesamation used the data by @dlopushanskyy from github.com/DmytroLopushan… + filled in a large block of missing country / region metadata for institutions with web-search enabled Codex.

@Hesamation Better version without arbitrary institution cutoff, some data cleaning and splitting contribution of each paper among institutions. China + USA dominant ofc, but looks a bit different, doesn't it?

someone analyzed all 5000+ accepted papers at ICLR 2026, and it's a good signal who's pushing the research of AI: > China has surpassed the US with 43.7% of the papers > Europe's contribution is surprisingly small (5.3% including UK)

@Hesamation Version using the same unique counting scheme (i.e. each institution is counted once per paper even if there's 50 institutions on a single paper). China's share grows. Maybe more cross-institution collaboration or larger projects involving many labs + industry?

@Hesamation The data is distorted, seems like you filtered out all institutions below 50 accepted papers? Many European countries are less centralized, so smaller paper counts from individual institutions

I’ll be presenting our work on Token Distillation for tokenizer transfer tomorrow at #ICLR! 🇧🇷 Com by our poster on Friday 3:15pm in Pavilion 3, Poster 113 Also come chat with me anytime about tokenizer transfer, embeddings, multilingual models or efficient architectures!

@A_K_Nain I‘m counting rephrasing-style synthetic data as augmentation here

@konstantdobler I would say even today it is very weak

Coming from the CV background, one pass over the data without heavy augmentation for pretraining was the most surprising thing to me.

📢I'm organizing a BoF session at #EACL2026 called Tokenization & Beyond, aiming to gather researchers exploring tokenization and alternatives such as byte-level and pixel-based approaches. Sign up using the form if you're interested! #NLProc @eaclmeeting

mAceReason-Math: - github.com/apple/ml-macer… - arxiv.org/abs/2603.10767

This is joint work with Simon Lehnerer, @FScozzafava, Jonathan Janke, and Mohamed Ali. Multilingual Reasoning Gym: - github.com/apple/ml-multi… - arxiv.org/abs/2603.10793

Releasing two multilingual reasoning resources from my internship at Apple: (1) 𝗠𝘂𝗹𝘁𝗶𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗚𝘆𝗺: reasoning puzzle generation across 94 tasks in 14 languages (2) 𝗺𝗔𝗰𝗲𝗥𝗲𝗮𝘀𝗼𝗻-𝗠𝗮𝘁𝗵: 140k translations of challenging math problems