Shom

@can some of sqlte's tests are public tho

in the light of this automated bun rewrite, SQLite’s open-source core, closed-source tests policy feels prescient but not exactly sure how

@jarredsumner Is this fuzzer a library or made by you?

There is now a fuzzer running 24/7 for every language parser in Bun, ranging from .npmrc files and .patch files to shell scripts to jsonc & typescript & css. Once it minimizes a repro, it sends to Claude to fix and then I review.

@boshen_c it's from zig presumably

So many allocators! We only have 1 in Oxc 😂

Thanks to the Rust rewrite, I now learn why Bun is fast First find: it uses a thread local arena for ASTs

This is growth-hacking dressed up in open-source language, @radixark please stop doing it immediately. Paying people in platform credits to star a GitHub repo and repost a marketing tweet isn't "fueling the community" — it's laundering paid promotion through the trust signals open source depends on. Stars are supposed to mean someone found a project useful. Attach a $200 bounty and the number means nothing. GitHub's own policies prohibit this for exactly that reason.

@zephyr_z9 full attention also has linear scaling...

Ok, so it's a linear attention variant

New modded-NanoGPT optimization benchmark result: @wen_kaiyue has improved upon both the Muon and AdamW baselines, by replacing their weight decay with hyperball optimization. The new record is 3325 steps.

@kalomaze if you spam on book scans you can get probably even more tokens

what i find most interesting about this release is that you can approach ~1T raw tokens from *before WWII*, before synthetic augmentation or rephrasers or whatever. that's the floor. and a post-WWII model could have the colloquial talk radio archives too, if transcribed...

@teortaxesTex i don't think they solved long agentic context reasoning, only some basic hashhop stuff...

Reminder: Magic dot dev had solved context 2 OOMs beyond today's DeepSeek 1.5 years ago. Now they're probably dealing with trillion-long sequences. AND YET! nobody cares about them. In America, this level of innovation is just noise. Open source/China have no hope of catching up.

The massive kv cache reduction of deepseek may unlock agent scaling as an economical choice...Imagine defaulting to 4 parallel agents solving one of your problem with each agents calling 10~20 subagents in parallel to explore different choices.

It's just a small piece of our bigger puzzle, to build a solid ecosystem for linear attention, and to make KDA as plug-and-play as flash-attn.

@_ueaj it may be due to increased parameters instead of increased kv cache rank from enlarged projections. What if you enlarge MLP layers for smaller kv cache rank models to balance the two models' params? Or do an mlp style expansion in projections?

New blog! You can just keep increasing the amount of heads in your model with no diminishing returns on ICL up to atleast 4x. For reference that would make the o_proj head dimensions in this experiment 16k x 2k. Additionally, if you perform a truncated SVD on full rank master weights to train MLA instead of training them as two separate matrices, you can recover most of the ICL capability but with less memorization. I think MLA specialized optimizers are a direction worth exploring and are very underserved rn. Unfortunately I have more important projects to attend to and I've burned like 300$ on compute for this already. I would highly recommend someone trying to scale this up and see how well we can do.

@fleetwood___ maybe it's too small?

The models, they just want to learn (their current task and literally nothing else). Training a toy transformer on 3 digit addition, sorting, reversal and modular addition. Complete lobotomy at every task transition.

@facontidavide I suggest checking the code rigorously for hacking perhaps by asking another code agent like codex to do it. They have very creative ways to game the benchmark and get high scores.

@ShomLinEd what do you mean? If by "hacking" you mean "cheating" the benchmarks somehow, I am going to say "no".

Claude Code just "invented" a new lossless codec on the Pareto frontier 🤯

@Dorialexander isn't gpt-4 rumored to have 200B activated params

Since scale pilled people are still at it, reminder there is in all likelihood no model deployed with more active parameters than 2020 GPT-3.

@TommyGun_AB @honeylemon0124 i am not able to find mentions of bee in IHNMAIMS hmm

Sad that we never really got to know why he was so obsessed with bees 🥲

@bigeagle_xd code廉价，验证不廉价

过去很多软件工程集中在“怎样方便维护、怎样可扩展、怎样尽量复用代码”等方面 如果code是廉价的，理想情况下，应该每个任务、每个场景直接原地写出最合适的软件

@ClementDelangue huggingface.co/datasets/nex-a… We published 70k high quality agent traces:)

We need more open agent traces datasets. Who can help?

The TurboQuant paper (ICLR 2026) contains serious issues in how it describes RaBitQ, including incorrect technical claims and misleading theory/experiment comparisons. We flagged these issues to the authors before submission. They acknowledged them, but chose not to fix them. The paper was later accepted and widely promoted by Google, reaching tens of millions of views. We’re speaking up now because once a misleading narrative spreads, it becomes much harder to correct. We’ve written a public comment on openreview (openreview.net/forum?id=tO3AS…). We would greatly appreciate your attention and help in sharing it.

flash-linear-attention is now seeing over 15,000 daily downloads. 📈 We @SonglinYang4 @uniartisan are honored to see fla becoming a piece of the core infrastructure for efficient model archs. Grateful to the community for the trust and support. github.com/fla-org/flash-…