Krish Ray
3.4K posts

Krish Ray
@KrishanuAR
Know thyself. Midwit. I work on AI/Model Risk.


Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize through composition. We observe a powerful property when training RLMs: for tasks with shared structure that look different, the root model naturally learns the same trajectory, meaning it views the two task trajectories as the same! In other words, the Transformer does not need additional generalization capabilities to transfer capabilities from one task to the other, the harness induces it. We find that well-designed harnesses form a quotient set over task trajectories, meaning their individual LLM calls can see structurally “similar” tasks as near-identical, token-for-token! Harnesses can effectively generalize for the Transformer during training, without relying on any intrinsic generalization capability from the model. For example, RLMs can see problems of different lengths as the same: we show that RLMs can train exclusively on short tasks, and fully generalize to similar but unseen tasks 8-32x longer because it produces near identical trajectories for both. Taking this further, we show that tasks across different domains (e.g. math solutions vs. essay writing) that share a decomposition strategy exhibit the same generalization effect. RLMs can train on the problem of finding which essays belong to the same author and improve performance on finding math problems that share similar solutions. The full blogpost, experiments, and discussion are in the thread below.




@deanwball wrote: >This future strikes me as a dystopian hellscape So, am I getting the chain of reasoning right here? In an open-weight-dominated world, private investment in foundational AI research is deterred because labs can’t capture as much surplus from selling inference. Open models undercut them, expected returns fall, and the cost of capital rises. AI research is capital intensive, so less private capital means slower progress. Public funding then has to step in, AI becomes public infrastructure, and we arrive at “full AI communism.” Okay. Being charitable here, we can assume Dean is well aware that much of modern scientific progress was seeded by federal science and grant programs. It would appear the underlying belief is that the private sector is a better accumulator of capital, that certain incentives are required for it to operate, and that open-weight models work against those incentives. That stance is probably true and hard to argue against. The part he seems to keep gliding over is: More capital ≠ progress. How much of this intuition comes from the applied/technology world, where putting more money behind a problem often does move things forward in a fairly direct way? Does basic science work the same way? There is seldom immediate ROI, and we often have no idea which line of inquiry will produce the next breakthrough. So is the objective really to concentrate as much capital as possible inside a few institutions, or to sustain as broad a search as possible across many of them? Bell Labs is probably the strongest example for Dean’s side. But what exactly is the lesson of Bell Labs? That private industry is naturally good at funding basic science? Or that basic science can thrive inside a private institution when it has enormous durable rents and the freedom to tolerate uncertain returns for a very long time? And even there, how much of Bell Labs’ success came from having a particular set of problems, people, and institutional assumptions concentrated in one place? Presumably every institution develops some version of that lens. The question is what happens when most frontier AI research is increasingly happening inside a handful of labs hiring from similar talent pools, using similar infrastructure, watching the same benchmarks, and facing similar commercial incentives. Maybe they are simply the institutions best positioned to find the next breakthrough. Though the DeepSeek moment should at least make us less confident about that. A lab in China, operating under very different constraints and with far fewer resources, found a path to a competitive model that the better-capitalized US labs had not pursued in quite the same way. How much should that update us toward thinking that breadth of search matters alongside depth of investment? Because once capable models diffuse, the search gets broader again. More researchers can modify them, build around them, and apply them to problems that the frontier labs have no particular reason to care about. Some of that work will feed back into AI itself, and some of it will show up in biology, chemistry, physics, medicine, engineering, or somewhere none of us are looking yet. So I understand the claim that open weights reduce the rents available to finance frontier training runs. I am less clear on how we get from there to open weights slowing technological progress overall. That seems to require treating hyperscaler capex as a fairly direct proxy for progress, when the whole question is how much progress comes from concentrating resources at the frontier and how much comes from broadening the number of people and institutions able to explore what the technology can do.



@deanwball wrote: >This future strikes me as a dystopian hellscape So, am I getting the chain of reasoning right here? In an open-weight-dominated world, private investment in foundational AI research is deterred because labs can’t capture as much surplus from selling inference. Open models undercut them, expected returns fall, and the cost of capital rises. AI research is capital intensive, so less private capital means slower progress. Public funding then has to step in, AI becomes public infrastructure, and we arrive at “full AI communism.” Okay. Being charitable here, we can assume Dean is well aware that much of modern scientific progress was seeded by federal science and grant programs. It would appear the underlying belief is that the private sector is a better accumulator of capital, that certain incentives are required for it to operate, and that open-weight models work against those incentives. That stance is probably true and hard to argue against. The part he seems to keep gliding over is: More capital ≠ progress. How much of this intuition comes from the applied/technology world, where putting more money behind a problem often does move things forward in a fairly direct way? Does basic science work the same way? There is seldom immediate ROI, and we often have no idea which line of inquiry will produce the next breakthrough. So is the objective really to concentrate as much capital as possible inside a few institutions, or to sustain as broad a search as possible across many of them? Bell Labs is probably the strongest example for Dean’s side. But what exactly is the lesson of Bell Labs? That private industry is naturally good at funding basic science? Or that basic science can thrive inside a private institution when it has enormous durable rents and the freedom to tolerate uncertain returns for a very long time? And even there, how much of Bell Labs’ success came from having a particular set of problems, people, and institutional assumptions concentrated in one place? Presumably every institution develops some version of that lens. The question is what happens when most frontier AI research is increasingly happening inside a handful of labs hiring from similar talent pools, using similar infrastructure, watching the same benchmarks, and facing similar commercial incentives. Maybe they are simply the institutions best positioned to find the next breakthrough. Though the DeepSeek moment should at least make us less confident about that. A lab in China, operating under very different constraints and with far fewer resources, found a path to a competitive model that the better-capitalized US labs had not pursued in quite the same way. How much should that update us toward thinking that breadth of search matters alongside depth of investment? Because once capable models diffuse, the search gets broader again. More researchers can modify them, build around them, and apply them to problems that the frontier labs have no particular reason to care about. Some of that work will feed back into AI itself, and some of it will show up in biology, chemistry, physics, medicine, engineering, or somewhere none of us are looking yet. So I understand the claim that open weights reduce the rents available to finance frontier training runs. I am less clear on how we get from there to open weights slowing technological progress overall. That seems to require treating hyperscaler capex as a fairly direct proxy for progress, when the whole question is how much progress comes from concentrating resources at the frontier and how much comes from broadening the number of people and institutions able to explore what the technology can do.



@deanwball wrote: >This future strikes me as a dystopian hellscape So, am I getting the chain of reasoning right here? In an open-weight-dominated world, private investment in foundational AI research is deterred because labs can’t capture as much surplus from selling inference. Open models undercut them, expected returns fall, and the cost of capital rises. AI research is capital intensive, so less private capital means slower progress. Public funding then has to step in, AI becomes public infrastructure, and we arrive at “full AI communism.” Okay. Being charitable here, we can assume Dean is well aware that much of modern scientific progress was seeded by federal science and grant programs. It would appear the underlying belief is that the private sector is a better accumulator of capital, that certain incentives are required for it to operate, and that open-weight models work against those incentives. That stance is probably true and hard to argue against. The part he seems to keep gliding over is: More capital ≠ progress. How much of this intuition comes from the applied/technology world, where putting more money behind a problem often does move things forward in a fairly direct way? Does basic science work the same way? There is seldom immediate ROI, and we often have no idea which line of inquiry will produce the next breakthrough. So is the objective really to concentrate as much capital as possible inside a few institutions, or to sustain as broad a search as possible across many of them? Bell Labs is probably the strongest example for Dean’s side. But what exactly is the lesson of Bell Labs? That private industry is naturally good at funding basic science? Or that basic science can thrive inside a private institution when it has enormous durable rents and the freedom to tolerate uncertain returns for a very long time? And even there, how much of Bell Labs’ success came from having a particular set of problems, people, and institutional assumptions concentrated in one place? Presumably every institution develops some version of that lens. The question is what happens when most frontier AI research is increasingly happening inside a handful of labs hiring from similar talent pools, using similar infrastructure, watching the same benchmarks, and facing similar commercial incentives. Maybe they are simply the institutions best positioned to find the next breakthrough. Though the DeepSeek moment should at least make us less confident about that. A lab in China, operating under very different constraints and with far fewer resources, found a path to a competitive model that the better-capitalized US labs had not pursued in quite the same way. How much should that update us toward thinking that breadth of search matters alongside depth of investment? Because once capable models diffuse, the search gets broader again. More researchers can modify them, build around them, and apply them to problems that the frontier labs have no particular reason to care about. Some of that work will feed back into AI itself, and some of it will show up in biology, chemistry, physics, medicine, engineering, or somewhere none of us are looking yet. So I understand the claim that open weights reduce the rents available to finance frontier training runs. I am less clear on how we get from there to open weights slowing technological progress overall. That seems to require treating hyperscaler capex as a fairly direct proxy for progress, when the whole question is how much progress comes from concentrating resources at the frontier and how much comes from broadening the number of people and institutions able to explore what the technology can do.

I think open-weight is basically as diffusion-accelerationist as closed (maybe slightly more on the margin), but considerably development-decelerationist due to raising ai hyper scale cost of capital. By the way, I don’t ascribe moral or ethical valence to these terms; I’m being descriptive.

I think open-weight is basically as diffusion-accelerationist as closed (maybe slightly more on the margin), but considerably development-decelerationist due to raising ai hyper scale cost of capital. By the way, I don’t ascribe moral or ethical valence to these terms; I’m being descriptive.


𝘀𝘁𝗼𝗽 telling Claude Code/Codex "no em dashes". 𝘀𝘁𝗼𝗽 telling Claude Code/Codex "stop saying delve". 𝘀𝘁𝗼𝗽 telling Claude Code/Codex "don't sound like AI". you never gave it a 𝘄𝗿𝗶𝘁𝗶𝗻𝗴 𝘀𝘆𝘀𝘁𝗲𝗺. every README, PR description and landing page ships in the same AI voice, and you ban words one at a time. orwell wrote the fix 𝟴𝟬 𝘆𝗲𝗮𝗿𝘀 𝗮𝗴𝗼. six rules, 1946. paste them into your global CLAUDE.md / AGENTS.md and every session picks them up: → never use a long word where a short one will do → if it is possible to cut a word out, cut it out → never use the passive where you can use the active 6 blocks you can copy-paste directly 👇







First of all, thank you for your patience and understanding. Our launch on July 3rd ran into issues with the website that prevented orders from going through — we're sorry for the frustration that caused. We've spent the extra time optimizing the website together with our hosting partner and increasing its capacity. While no online launch is ever completely predictable, we've done everything we can to prepare for the demand. INDX Development Kit Orders Open Wednesday, July 15 at 15:00 CEST The INDX Development Kit is built for makers, developers, and innovators who want to explore what's possible with the INDX ecosystem. Whether you're planning your first multi-tool setup or already have exciting ideas in mind, we're looking forward to seeing what you'll create. Since the original Founders pricing was announced, component costs have increased, and we've continued to develop both the Tool and the Toolhead with additional features and improvements. As a standalone product, the Development Kit therefore has its own updated pricing. Delivery is expected to begin during the first half of September. Thank you for your patience and for being part of this journey. Your enthusiasm, encouragement, and support have meant so much to all of us, and we can't wait to get INDX into your hands. The Bondtech Team










Introducing Kimi K3: Open Frontier Intelligence 🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost 🔹 Built for long-horizon agentic coding and self-evolving workflows Kimi K3 is now live on on Kimi.com, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026. 🔗 API: platform.kimi.ai 🔗 Tech blog: kimi.com/blog/kimi-k3




