Xiaoliu.x

Sometime dreaming is a thinking game, openreview.net/forum?id=HHsD9…

Why do people keep showing off something that others want to participate in, but can't actually join? For example, SpaceX. Maybe I should just go back home and invent aliens!

@jeremyphoward Check the GLM 5.2

I disagree with this decision and I don't like it. But also... HOW DID ANTHROPIC NOT SEE THIS COMING‽ It is *the* obvious response to "this is too dangerous for anyone except us to use", since that relies on a premise ("we are uniquely good") that almost no-one agrees with.

@TheAhmadOsman Opensource AI the final frontier...

@jbhuang0604 That's legendary, lol.

Explaining JEPA in 10 seconds

The Delta Memory design for hybrid online memory is very inspiring. It presents a great direction for post-training. We can leverage RWKV memory to bring RNN features into high-level HRM. github.com/xiaol/HRM-RWKV…

Pure RNN can code: RWKV-7 G1g 7B (100% RNN) batch vibe coding on huggingface.co/spaces/BlinkDL… select [✨HTML Generation] to try any prompts including your own (click [Stop] a few times before trying a new prompt - sometimes laggy as it's using ZeroGPU). Not bad for a base model🙂

@hthieblot 爱

pitch me your company in 1 word.

With the Universal Transformer setup and recurrent state rollouts built into the architecture using triple latent states, how can we build an even stronger model? arxiv.org/abs/2604.25930

ArXiv limits researchers’ creativity: if you give them eight submissions over two months, only three end up published.

For RL, we’re investigating how to adopt high-concurrency states to implement horizontal & depth rollouts, and breaking OOD constraints stands as the key bottleneck researchers need to resolve to boost model capacity. arxiv.org/abs/2604.09671

Two months ago, higher-order design within attention mechanisms was seen as a promising research direction. However, clever iterative loops now perform better for small models. My work at arxiv.org/abs/2606.05175 centers on more intricate high-dimensional high-order spaces, exploring how to embed these structures into LLM latent spaces in a more intuitive manner.

Sapient's HRM text pushes pretraining to the next level. I hybridized it with RNN. It looks like the H level replacement for RWKV has more potential to explore. I'm thinking of doing a full pretraining.

RWKV community built a SKILL to migrate from GDN-2 (and other similar archs) to RWKV-7, with interesting results 🙂 let us know your experience github.com/Jellyfish042/G…

Let us to find a new way to scale. Looped Transformer with linear attention is impressive. If we combine it with the core ideas of RNN, we can gain advantages from both pure and hybrid architectures.

RWKV-7 G1g is here: the world's best pure RNN LLM, and a competitive LLM in general. Try huggingface.co/spaces/BlinkDL… for bsz16 7B inference. G1h in June 🙂 p.s. const 15000+tps decoding on single 5090: github.com/BlinkDL/Albatr…

Gated DeltaNet-2 is almost exactly RWKV-7's DPLR recurrence, not acknowledging the elephant in the room 🙂

How can I compare Transformer with modern RNNs? @GoodfireAI

Current LLMs are trapped in the dilemma of incremental inovations.