Ethan Hersch

Was great working on this project with @stai_research, @StanfordAILab, @BrandoHablando, @ObbadElyas, @srivatsavad, @kirillacharya, Jolene Wang, @zhankezhou, @StevenDillmann, @yegordb, and @sanmikoyejo. Congrats!

8/8: Open source 🤝 Certify the judge — don't re-label it. 📰Paper [arXiv soon]: github.com/ehersch/CertJu… 💻 Code: github.com/brando90/cert-… Poster: github.com/ehersch/CertJu… ICML 2026 Math-AI workshop. Feedback welcome!

7/8: Downstream + scope 📈 The certified judge ranks theorem-generation agents on VeriBench: DSPy ReAct 0.615 > Trace++ 0.588 > Trace+ 0.477 > baseline 0.470. Scope, stated honestly: validation-set calibration evidence — not universal reliability. A locked held-out test is next.

🚨🚨Announcing CertJudge🚨🚨 But who judges the judge? … New work from Stanford University, Harvard University, Hong Kong Baptist University

6/8: Catches what agreement misses 🛑 A deliberately broken judge still hits ≈0.45 human Spearman — yet fails weak monotonicity (12.5%). Known failure mode: aggressive α-renaming scores 0.399 vs ≥0.849 for milder rewrites. Judges key on surface strings the kernel ignores.

5/8: Results 📊 TI_var predicts judge-level human alignment: validation-set Spearman 0.833 (avg labels), 0.905 (pass1), 0.738 (pass2) — across GPT-5.3-Codex, Claude Opus 4.7, and Qwen3-Coder judge families.

I wanted to understand post-training by actually doing it‼️ So I took a Qwen2.5-0.5B base model and walked through SFT, DPO, and GRPO, with notes on how these methods fit into the post-training pipeline. Hopefully I can teach you something: ehersch.github.io/Post-Training/

Reproduced CLIP from scratch for ~$5 💸 Two pretrained encoders (ResNet and DistilBERT), symmetric InfoNCE loss, 40 epochs on Flickr30k. It was a cool experience Blog post: ehersch.github.io/CLIP-blog Check it out!

@reinforce2020 @TheChiefNerd When did he say this?

@TheChiefNerd That’s great, but 6 months ago Jensen Huang told students not to study Computer Science. Do his statements have any informational content, or just market positioning?

🚨 Nvidia CEO Jensen Huang: “The narratives of AI destroying jobs is not going to help America — It's false … Somebody said that AI Is going to destroy all of the software engineering jobs … We now have agentic AI inside Nvidia … The software engineers are busier than ever.”