Dan Ofer (Was @ICML,@Worldcon )

1/ Our paper “Protein Language Models Expose Viral Immune Mimicry” is now published in Viruses! We show that protein Language Models can identify viral proteins, and those that fool our immune system.

@prajdabre Look up gradient caching? JEPA approach and smaller teacher model + aggressive noising? Bottleneck autoencoder?

ML interview question: Suppose you are implementing Knowledge distillation, and you have a teacher and a student model. However you simply do not have the necessary GPU resources to fit both the teacher and the student into the GPU at the same time. What is your solution?

@lucasmaes_ Neat. Gonna try it vs other JEPAs :D

JEPA are finally easy to train end-to-end without any tricks! Excited to introduce LeWorldModel: a stable, end-to-end JEPA that learns world models directly from pixels, no heuristics. 15M params, 1 GPU, and full planning <1 second. 📑: le-wm.github.io

@askalphaxiv hah. I randomly added that in my current jepa (unpublished work). I didn't even realize that was different than the existing ones

Yann LeCun and his team dropped yet another paper! "V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning" In this V-JEPA upgrade, they showed that if you make a video model predict every patch, not just the masked ones AND at multiple layers, they are able to turn vague scene understanding into dense + temporal stable features that actually understands "what is where". This key insight drove improvements in segmentation, depth, anticipation, and even robot planning.

@pranamanam Sounds like they adopted the new "standards" of NLP - "llm as a (crooked) judge" 😂

1. Let's get this out of the way: there's no experimental validation. Those "sub-nM" claims are irrelevant. They "generate" sequences, fold with Boltz-2, and score with their own model. Not a single peptide was actually tested. 🙅‍♂️

I usually don't like to criticize papers on social media, but this one deserves it. Not familiar with @Ligandal, but so many problems: AI-hallucinated citations, figures, no real validation, not "structure-free", and definitely not diffusion. I'll go thru my criticisms below. 👇

@Yampeleg @chetaslua I prefer pelicans

@chetaslua this should be a benchmark wow

My first test of MiniMax M2.7 this made 3d fibre physics with all the specs of m2.7 is written

@Yampeleg For planning it's excellent and creative. Bouncing plans or optimizations between opus and gemini works REALLY well

Fight Opus vs Codex no problem, it’s an interesting debate. But just so you know, you are all sleeping on Gemini.

@Yampeleg All the labs are doing this tho; from prompt engineering on to autoresearch ("make loss go down")

The model was evaluated by how much it contributed to building the next version of itself. This is a crazy post. They basically did auto-research IRL: Maximizing how much the RL team's work is delegated to the model during it's development. (Answer: 30-50% btw) Everything researchers do: - Literature review - Experiment design - Data pipelines - Monitoring - Debugging - Code fixes - Pull requests .. Actively trying to delegate more and more between the research iterations. Crazy crazy crazy times

@Scientific_Bird @Aella_Girl Was he a Bridge Builder?

@Aella_Girl I don't know how large role this has played, but it is relevant and fun. In "Guns, Germs and Steel," Jared Diamond tells an anecdote of a man contradicting a disease from having sex with a sheep.

Has bestiality always been historically so taboo? Why? It doesn't result in pregnancy; did it result in diseases? Why did humans evolve to get so repulsed from touching genitals with other species?

@Yampeleg עד שהתלהבתי מהK2 + resAttent

MiniMax-M2.7 woww

@karpathy But, does it run Doom?

Thank you Jensen and NVIDIA! She’s a real beauty! I was told I’d be getting a secret gift, with a hint that it requires 20 amps. (So I knew it had to be good). She’ll make for a beautiful, spacious home for my Dobby the House Elf claw, among lots of other tinkering, thank you!!

@BlancheMinerva Someone cracking an encrypted message and telling me the contents within that qubit time. (Oh wait, wrong domain....)

If I was going to claim that a finetuning methodology for machine unlearning “really worked,” what evidence would you like to see?

I too have joined the autoresearch + {trendy DL} thing crowd. Just hoping I can get a paper out on it sometime :)

@DataChaz @karpathy Sounds neat, once someone does some security on it - seems horribly exposed

THIS is the wildest open-source project I’ve seen this month. We were all hyped about @karpathy's autoresearch project automating the experiment loop a few weeks ago. (ICYMI → github.com/karpathy/autor…) But a bunch of folks just took it ten steps further and automated the entire scientific method end-to-end. It's called AutoResearchClaw, and it's fully open-source. You pass it a single CLI command with a raw idea, and it completely takes over 🤯 The 23-stage loop they designed is insane: ✦ First, it handles the literature review. - It searches arXiv and Semantic Scholar for real papers - Cross-references them against DataCite and CrossRef. - No fake papers make it through. ✦ Second, it runs the sandbox. - It generates the code from scratch. - If the code breaks, it self-heals. - You don't have to step in. ✦ Finally, it writes the paper. - It structures 5,000+ words into Introduction, Related Work, Method, and Experiments. - Formats the math, generates the comparison charts, - Then wraps the whole thing in official ICML or ICLR LaTeX templates. You can set it to pause for human approval, or you can just pass the --auto-approve flag and walk away. What it spits out at the end: → Full academic paper draft → Conference-grade .tex files → Verified, hallucination-free citations → All experiment scripts and sandbox results This is what autonomous AI agents actually look like in 2026. Free and open-source. Link to repo in 🧵 ↓

@ChemVagabond @_DimensionCap @karpathy @modal I can confirm this worked on my own set of (33) benchmarks. Funnily enough, thermostability is one of the ones where it didn't improve :D neat!

We @_DimensionCap ported @karpathy's autoresearch framework to biology. We let Claude run 50 experiments over the weekend on protein thermostability prediction via @modal. It beat a recent baseline (TemBERTure) using a 20x smaller model. Code + research blog later this week!

@Yampeleg Get Pubmed next time :)

I once downloaded ALL of arXiv to run RAG on it but the Iranians took out the server with a missile lol

@karpathy Ugh, making it plan experiments to run as a script works surprisingly badly (to many crashes, not enough updated then subagents start "cheating" re time budget)

I packaged up the "autoresearch" project into a new self-contained minimal repo if people would like to play over the weekend. It's basically nanochat LLM training core stripped down to a single-GPU, one file version of ~630 lines of code, then: - the human iterates on the prompt (.md) - the AI agent iterates on the training code (.py) The goal is to engineer your agents to make the fastest research progress indefinitely and without any of your own involvement. In the image, every dot is a complete LLM training run that lasts exactly 5 minutes. The agent works in an autonomous loop on a git feature branch and accumulates git commits to the training script as it finds better settings (of lower validation loss by the end) of the neural network architecture, the optimizer, all the hyperparameters, etc. You can imagine comparing the research progress of different prompts, different agents, etc. github.com/karpathy/autor… Part code, part sci-fi, and a pinch of psychosis :)

@championswimmer @tobi Wait, really?

@tobi Why is every test failing?

OK, well. I ran /autoresearch on the the liquid codebase. 53% faster combined parse+render time, 61% fewer object allocations. This is probably somewhat overfit, but there are absolutely amazing ideas in this.

@CamrynsOp @Aviation_Intel I was in one of those wargames! (~15+ years ago) They even had a really good solution for it from an American. Who has since retired because of Trump. :|

@Aviation_Intel CENTCOM has been planning for the precise scenario for decades… sorry, but I don’t believe that story a bit

The stories that the admin/pentagon was blind sided by Iran closing the strait is the most outrageous oversight I have ever seen in war planning if true.

Go home AI, you're drunk

@yoavgo Cognitive off loading. Along with horseless carriages

lol she is relentless