JD Kim

gpt-5.6 sol이 내 task에선 제일 잘 한다. RLVR 데이터 합성 중.

@thsottiaux 코덱스만 사용하는데 언제 워크를 사용해야하는지 모르겠다.

Here you are! Thinking I am about to announce a reset. But no. I’m just scrolling twitter and looking for feedback on ChatGPT Work. What should we improve?

@thsottiaux holy!!

To celebrate the launch of GPT-5.6 Sol, we will reset the rate limits again (twice) across ChatGPT Work and Codex over the next 24 hours. We want you to have the time to truly try ambitious tasks and get the hang of it. Happy exploring!

@thsottiaux @_rajanagarwal god tibo

Enjoy a full reset of your usage limits for ChatGPT Work and Codex. Propagating in the next hour. @_rajanagarwal just joined to work on model research and push on coding capabilities. You can thank him for pressing the button today.

Tracing back the history of Policy Gradient... Turns out back in 1992, they literally built a table for every single (s, a) combination and updated it using gradient ascent. Mind blown. 🤯 Since it's inherently non-convex with a noisy objective, there's no way it would've converged well on complex problems. Man, how long has this little algorithm been waiting for Deep Learning? 🥹

@ClaudeDevs Why is self-hosting not supported?

Artifacts in Claude Code are now also available on Pro and Max plans. Ask for an artifact, Claude writes the code, publishes it live to claude.a‍i, and updates it in real time while it keeps working. Pages are private to your account and fully self-contained.

@mgoin_ 한국어 지원도 부탁해요!👏

GLM 5.2 DSpark update! The full Speculators training run is well underway and we have the epoch-1 checkpoint ready for your GPUs using vLLM nightly: huggingface.co/RedHatAI/GLM-5… This improves upon the speedup from the preview checkpoint by another 1.5-2x. Stay tuned for more!

(1/N) Earlier this year we showed that RL works surprisingly early in LLM pretraining (x.com/rach_it_/statu…). Now we have completed our full excursion. Our new preprint (arxiv.org/abs/2606.04272) challenges a lot of conventional wisdom about the role RL plays in LLM training!

@jietang critic model for PPO

Any new features we must have in the next version of glm?

@waterloo_intern ABCDA 😂

After reading up a bit on ML research post transformer era, I was upset that it seems to have converged on hyper-optimizing matmul-based algorithms: (MHA, MQA, MLA, SWA, DSA, GQA, SWA-GQA, ABCDA [only one of these is made up]). Surely, an algorithm that is not Attention based is sitting there waiting to be discovered. > the researchers are just being lazy but this is a stupid conclusion. How can you blame researchers, when the hardware they train on is optimized for matmuls (tensor cores / systolic arrays). Any algorithm not a matmul is literally bound to die, even if it's twice as good as attention. Add compute constraints, you have to be crazy to research any direction not attention based (basically @sarahookr 's hardware lottery essay) We talk about hardware-software co-design in inference, but it seems that, to get to the next leap in research, we'll need hardware-research co-design. At first, it seems this will never happen, given typical multi-year hardware tape-out constraints. But then you look at @OpenAI. 9 month tape-out. Better "training" and serving . Why fab your own chip if it's just going to be systolic-array based? Why not just buy Nvidia? > "But Nvidia GPUs are scarce" Then buy TPUs/AMD/Qualcom/Cerebras. Sure the software is not that good, but if you're OAi, you can hire an army of engineers to unlock the full capability. Either they moved away from attention and have a new algorithm they needed their own chips to train it on (unlikely given that a 9-month tape-out with a TPU vendor implies reusing IP)...or research is dead and we're never escaping attention / matmul based algo.

Introducing a limited preview of GPT-5.6 Sol, our next generation frontier model, as well as GPT-5.6 Terra, a balanced model for efficient, everyday work, and GPT-5.6 Luna, a fast and affordable model for high-volume work. openai.com/index/previewi…

We trained a SoTA model on @harvey's Legal Agent Benchmark! Graphs like this are what make life sweet :)

I think that an agent’s action can be abstracted as all the tokens output by the model in the role: assistant turn.

@kalomaze @teortaxesTex this method does not solve the group-wise optimization problem. In log-horizon tasks where compaction is available, it is difficult to group them because traces of highly variable lengths are produced even from the same prompt

@teortaxesTex >forces the assumption of a frozen reference model uuuugghhhhhhhhhhhhhhhh

Maybe a better alternative to GRPO than simply falling back on PPO

So true

@slime_framework Any plans to add MOPD support to the roadmap too? Would love to see that!

Thanks for the support! A small note: slime has supported not only OPD, but the full RL + OPD post-training workflow since GLM-4.5. More to come for scalable agentic RL infra.

@llllvvuu Interesting. Another angle: they might have used a GRM rather than a traditional value/critic model. If so, that would naturally support turn-level outcome rewards instead of having to assign credit at the token level.

I’m not going to comment on the future of PPO vs GRPO, but I find the specific line of reasoning “critics stabilize long-horizon learning” a bit unrigorous. Often it’s justified with some informal language like “critics perform credit assignment, which means only the tokens that matter get trained on”. But if you actually try to formalize this, the statement is much closer to “TD learning increases tractability of long-horizon learning” which is both obvious and orthogonal. The more relevant question then is not “did they estimate advantage using MC or function approximation”, but rather “what did they set lambda to in GAE”, which is a much more interesting question. If you set lambda = 1, it’s not clear that the critic is a groundbreaking improvement. You’re exposed to reward-to-go noise either way. This seems more like a property of policy gradient than of MC vs function approximation. On the other hand, lambda < 1 is a bet against scale, because bias doesn’t wash out with scale the way that variance does. @MillionInt talks about this. This is why, for example, GRPO does in fact work across 1000s of turns and 10s-100s of compactions. If variance scaling with horizon is your biggest concern (not obvious that it should be), there are algorithms that are actually unaffected by horizon. Evolutionary Strategies (ES) comes to mind, which has variance scale instead with parameter subspace dimension (e.g. low-rank mutations have lower dimension).

AI is not an entity that can be held accountable or absolved of responsibility. If a major failure occurs, saying "the AI did it" changes nothing. Ultimately, humans must take responsibility and guarantee the outputs of AI. Just as we do not sign contracts we do not understand, there must be a mutual understanding and agreement regarding the decisions and outputs of AI. Ultimately, the role of humans that will not disappear is not to make better decisions than AI, but to understand and verify the decisions made by AI, and to sign off on the agreed-upon matters.

It reads like purism vs pragmatism, which is what makes it fun. As someone doing LLM RL for a living, my taste is MAI. But purity has a price. Given limited resources and full authority to build an agentic model, would I really have made the same call?

Nemotron goes the other way. It leverages open-weight models heavily, using synthetic data aggressively to measure and improve data quality. Same on the RL side: PivotRL trains only on the most valuable trajectories — faster and more efficient than E2E RL.

Two technical reports landed around the same time recently: MAI-Thinking-1 (Microsoft) and Nemotron-3-Ultra (Nvidia). Both detail how they build modern agentic models — data, RL reward design, infra. What’s fun is how opposite their philosophies are.