Jonas

@hxiao 99% vibecoded with claude, crazy what is possible today explore it yourself here: jonasloos.github.io/embedding-ring…

@hxiao Nice project! I think PCA may not be the best basis to judge this. Fitting the circle directly shows that some embeddings seem to encode it much better than pca suggests. link ↓

take a 3D Pikachu model, render 36 frames rotating 360°, encode each frame with different vision embedding models (incl GeminiEmbedding 2), then project to 3D with PCA. If a model truly "understands" that these images are the same object rotating, the embeddings should form a circle/ring - smooth, continuous, preserving the angular order.

@fchollet invention seems like just another task for the automation machine

If you build an automation machine, the way to monetize it is to sell it to as many people as possible -- anyone who has tasks to automate. But if what you build is an invention machine, then the best way to monetize it is to use it yourself.

@dogecahedron I could imagine more, but one would have to test. But maybe there are also cases where LLMs would profit from more redundancy/explicitness

@LoosJonas modern languages introduce some amount of redundancy in order to make it possible to localize typos with statical analysis. Since LLMs are way less prone to errors you could probably do better tradeoffs here. however this soule probably not save more than 10% of tokens

we need a better programming language optimized for coding agents

@dogecahedron yes

@LoosJonas token efficiency after accounting for a good tokenizer sounds like we are looking for a language that assigns short encodings to useful programs🤔

@dogecahedron Token-efficiency, adjustments for the autoregressive nature of llms, less long-range dependencies, ... But yes, the bias towards common languages is a disadvantage for new languages

@LoosJonas llms seem somewhat syntax agnostic? how could new syntax outperform common languages that have massive training sets?

@enjoyingthewind @carl_sterns @robertluxemburg @josephdviviano tru

@carl_sterns @robertluxemburg @josephdviviano Yeah and ChatGPTs are very clearly recognizable as well x.com/LoosJonas/stat…

me: "can you use whatever resources you like, and python, to generate a short 'youtube poop' video and render it using ffmpeg ? can you put more of a personal spin on it? it should express what it's like to be a LLM" claude opus 4.6:

@robertluxemburg @josephdviviano Quite nice that it just works (also gpt5.4, same prompt)

@josephdviviano interesting prompt. here is chatgpt 5.4:

@dogecahedron Better verification, security, syntax, agent-tooling, ... Good thing is that we dont have to guess, but could just evaluate how well coding agents can use it and optimize

@LoosJonas what are the missing specs?

@JesseFarebro Chatgpt/Claude/Gemini/... make it really easy to find prior related work

It is infuriating how many ICML submissions could have been entirely prevented if authors just took 5 minutes to do a literature review. Ignoring ~10 years of established work on the exact idea you are proposing is just lazy.

@SwayStar123 Congrats

Now that I'm working in a frontier lab i can finally vague post about research Just tested out a random idea i had and its significantly better than baseline

@Dorialexander @amiribtc Wouldn't emergence then just be the point at which the signal starts to be capturable?

@amiribtc not completely but rather the by-product of very noisy training data: signal is there but weak and you need a large nest to catch it.

Hard disagree. Everything is there at small scale, just less capable.

@N8Programs I remember this working in the past, but recently had some cases where the model just refused to think more than 5s, no matter what I tried

@LoosJonas set to extended thinking (or heavy if you're on pro) and say 'please think as long as you can/think hard'. if you don't get a 'thinking' popup retry.

one thing that's mildly infuriating with gpt-5.2 (when it doesn't think long enough) is how it responds to autoregressively trapping itself. it often says 'you got this wrong/this isn't fully correct', works out the math, realizes you are right, and just restates your conclusion without saying 'whoops i got this wrong'. this ragebaits you (or at least ragebaits me) because it goes 'you are wrong [does work] and thats how you fix it [thing you originally did]'.

@VictorTaelin Can you recommend ts for programming language implementations?

Another day, another mindset. Dismissing Codex CLI and Claude Code, back to the "just put everything in context, build yourself from ground up, use AI just to fill gaps", and my love for Opus 4.6 is suddenly back. Fast mode is incredible! Just implemented a toy programming language in 30 minutes. All I had to do is write this 60-loc prompt, and Opus gave me a fully working implementation in ~30 seconds, including parser, stringifier, main. Everything works. (Just don't ask why I need this.)

@SwayStar123 And still, they're all over the timeline

Criminal twitter management 1. No image even though they have an amazing chart in the blog 2. Links in the tweet 3. No information about the product even though they are a no name company releasing something awesome

@seconds_0 Interactive sub 1 second app/website generation; chat models doing preliminary deep research while you're still typing/speaking; ...

The most important part of the Taalas demo is the revelation that superhuman OOM token output is simply possible Theres now a direct path through time for 15k tok/sec gpt5.3 codex xhigh the world that has that is very confusing indeed

@dravidan @seconds_0 Image tokens can be generated in parallel, so it's already possible, see e.g. genie 3

@seconds_0 At what tokens/sec are we in the realm of live video generation —- TP math: ~2000 tokens per image, so that’s like 7 FPS; is the math mathing ?

@DavidSHolz arxiv.org/abs/1612.03238

this remains one of my favorite papers of all time

@karpathy we need a benchmark evaluating programming languages for coding agents. And then use it create new ones

I think it must be a very interesting time to be in programming languages and formal methods because LLMs change the whole constraints landscape of software completely. Hints of this can already be seen, e.g. in the rising momentum behind porting C to Rust or the growing interest in upgrading legacy code bases in COBOL or etc. In particular, LLMs are *especially* good at translation compared to de-novo generation because 1) the original code base acts as a kind of highly detailed prompt, and 2) as a reference to write concrete tests with respect to. That said, even Rust is nowhere near optimal for LLMs as a target language. What kind of language is optimal? What concessions (if any) are still carved out for humans? Incredibly interesting new questions and opportunities. It feels likely that we'll end up re-writing large fractions of all software ever written many times over.