Marmotinha Marotinha

@baltabaev Pavel, a curiosity: Is there a result that could ensure that the calculation is right? That's how I often evaluate engineering agentic works - metrics and results. When stuff is too very complex, being able have strong evaluation in the results does help a lot.

I’ve spent well over 10,000 hours studying math in my life, yet I can’t understand these proofs, at least not without weeks of digging deep into each topic. What’s more, none of my math PhD friends know much about these problems either, and they can’t verify most of them without working directly in the field (yes, math is VERY diverse). LLMs are getting smarter than the experts themselves, and I’m not sure we have enough bright human minds to verify everything that will come out of them in the coming years. Remember when we compared AI intelligence to PhD students? I think we’re past that.

@grok @thesupermanmx how promissing it is, @grok ?

Yes. Google Research published "Memory Caching: RNNs with Growing Memory" (arXiv:2602.24281, Feb 2026). It caches RNN hidden-state checkpoints so effective memory grows with sequence length, trading off linear efficiency against Transformer-level recall. On in-context tasks it closes most of the gap, though Transformers still lead in raw accuracy. The technique is real; "ending the Transformer era" is promotional overstatement.

Google has quietly published a method that might end the entire transformer era. For a decade, every major AI model on earth, ChatGPT, Claude, Gemini, has been chained to the exact same architecture. The Transformer. It changed everything by letting models look at entire sequences at once. But it has a fatal flaw. Its memory consumption scales quadratically. $O(L^2)$. As context windows grow to millions of tokens, the compute cost explodes. It requires massive data centers, thousands of GPUs, and fortunes in electricity. For years, researchers tried to replace Transformers with old-school Recurrent Neural Networks (RNNs) to get lightning-fast linear speed. The problem? RNNs have fixed-size memory. When texts get long, they forget. They choke on complex reasoning and recall tasks. So everyone went back to Transformers. Until Google dropped a bombshell new paper. They introduced Memory Caching (MC). Instead of forcing an RNN to rely on a fixed, static memory, Google’s technique lets the model's effective memory capacity grow dynamically with the sequence length. It acts like virtual memory for neural networks. By caching checkpoints of hidden states and integrating smart mechanisms like sparse selective retrieval, the model dynamically looks back at what it needs, without the crushing mathematical overhead of a Transformer. The results completely break the old rules of sequence modeling: • Matches Transformer-level accuracy on complex in-context recall tasks. • Completely closes the performance gap that kept RNNs sidelined for years. • Maintains linear complexity, making long-context processing radically cheaper and faster. We've spent years believing that massive, heavy attention mechanisms are the only way to build intelligent systems. Google just proved you can have the infinite memory of a Transformer with the blistering speed and efficiency of an RNN. The era of brute-forcing scale with quadratic attention is hitting a wall.

@DCinvestor Oh dude... the era of the nerds seems to be starting now. The first ones to get advantage from all of it are the ones who knows how to think. Maybe later, everyone else.

the era of the nerd is unfortunately over go the gym and start lifting

@kimmonismus It would really be exciting is if all this things gets available for everyone - not only for the rich and powerful. There is so many things that AI can help to solve too, such as poverty, misery, people dying for lack of food or basic stuff. That possibility thrills me.

Many people outside our AI community have absolutely no idea what's happening right now. I keep thinking about Demi Hassabi's words that we're entering the golden age of science. This is proof of his prediction. What will this affect? ​​Everything. Materials research, energy production, drug discovery. Everything. I don't want to get too caught up in the hype. But the fact that we could seriously cure every form of cancer, if not stop aging, at least double the normal rate, gain access to new energy sources, and so on, excites me.

something is happening I guess. gpt-5.6-sol is EXTREMELLY slow right now!

@Ananth7e @grok , what is this person saying? as far as I heard gpt "astra" is a model that is not yet release? or he/she is talking about something else?

openai astra has become borderline unusable. luna is way better at this point.

@blueemi99 For god sake, why would anybody want it back? Whats up with you buddy! hahaha

Didn’t they bring 5-hour limit back?

@trikcode we all agree we would uae it way way more. but once you meet Astra you'd change your opinion, probably.

I don’t need GPT-6. I need GPT-5.6 Sol at DeepSeek pricing.

@thsottiaux release astra today so we can have break throughs as well

The week was for efficiency. The weekend is for 10 major breakthroughs in science. There will be signs.

@gdb Release it man? Release it now for god sake!

ten significant advances in mathematics and theoretical computer science. solved using an internal version of Astra, our next major model, for a total cost of about $2000 at Sol API prices:

@elshayib_ Buddy, depending on what you're doing, deepseek-v4-flash will not do it... gpt-5.6-sol a great great model. No doubt.

Forget about codex resets, I topped up my deepseek account with 2$ yesterday I only slept 6 hours and the rest was work and still got 0.70$ 🤣, Deepseek V4 flash is free.

@kr0der did you mesure it somehow, or just feeling it? because sometimes gpt-5.6-sol feels faster even not in fast mode.

it seems like /fast in Codex got faster (used to be 1.5x speed, now up to 2.5x) for the same cost (2.5x higher than non-fast) honestly might be worth toggling it on now

@thsottiaux Now, to keep me awaken all night, throw Cerebras usage! 😍

To celebrate a week of efficiency and let you run 100'000 Luna threads this weekend... that's right... wait for it... I have reset usage limits for Codex and ChatGPT Work. Enjoy.

@thsottiaux Yesss! I was in 2%!!!

@jun_song will wait for china's GPUs

Local AI is the future. Get your hardware right now.

@thsottiaux 3% of my weekly usage in 10 minutes of "optimization"

What is happening @thsottiaux , I've just updated codex app and now my latest thread can never finish "optimizing conversation" and draining usage...

@ashpreetbedi It might depend in a lot of things such as gains in quality that are not well measured, or other stuff like company is growing and being able to keep the same team. Or, it could mean that their team is not as productive with AI as they think they are. It will on many variables.

Everyone knows I'm super AI-pilled. But one trend I'm noticing as I talk to more and more companies: the personal productivity gains are not translating into organizational growth and efficiency as expected. It's an odd dichotomy. Individuals are more productive. Engineers are writing an insane amount of code. But the gains aren't showing up in the numbers yet. Not sure how universal this is.

@Olao3w hey hey!