Otto Gradkowski

@NousResearch Bro, Nous Research is cookin.

Today we release Token Superposition Training (TST), a modification to the standard LLM pretraining loop that produces a 2-3× wall-clock speedup at matched FLOPs without changing the model architecture, optimizer, tokenizer, or training data. During the first third of training, the model reads and predicts contiguous bags of tokens, averaging their embeddings on the input side and predicting the next bag with a modified cross-entropy on the output side. For the remainder of the run, it trains normally on next-token prediction. The inference-time model is identical to one produced by conventional pretraining. Validated at 270M, 600M, and 3B dense scales, and at 10B-A1B MoE. The work on TST was led by @bloc97_, @gigant_theo, and @theemozilla.

@MaestroGaeta @ElijahRavitz @mnolangray @chrislhayes The article makes so many claims with basically no data backing the arguments. And the fact that its core assumption is that only demand drives pricing, and not supply, tells me there’s insane bias or ignorance here. Super weak, politically motivated analysis.

@ElijahRavitz @mnolangray @chrislhayes Yawn. Yimbys aren't s h i t. eastbayexpress.com/why-is-there-a…

For 45 years, Berkeley built virtually no new housing. By the mid-2010s, it was the most expensive college town in America. Shortly thereafter, YIMBYs took over and kicked off a building boom. Today, nominal rents are below 2018 rates—remarkable progress on affordability.

@benedictk__ Down

Germany / Switzerland - who wants to be on this?

@ihtesham2005 @garrytan This is still just optimization. Who will decide the objective function and constraints? Not to talk about a research program which involves many objectives, hypotheses, and validations.

If you still think AI agents can't do real research, this paper will end that argument. Researchers from Google and Meta built a framework where Claude Code proposes its own algorithms for making LLMs reason better, then tests them, then refines them based on what failed. No human in the loop after the environment is set up. In 5 rounds the agent discovered a controller with 4 coordinated mechanisms working together. EMA momentum stopping. Coupled width-depth control. Alignment-aware depth allocation. Conservative branch abandonment. The paper says directly: "a level of coordinated complexity that would be difficult to arrive at through manual intuition alone." That's a polite way of saying the agent built something a human probably wouldn't have. The cost of the entire discovery was $39.90. The cost of one researcher's coffee budget just outperformed years of hand-tuned work. Paper is from Google and Meta. Read it here: arxiv.org/abs/2605.08083

@daniel_w_owens @garrytan The American obsession with banning smokers is retarded. Here in Switzerland it’s normal to allow smokers their place outside of a building. Not being able to handle the smoke is silly, just keep walking.

I can’t stand the smell of smoke, so I don’t have a dog in this fight, but I gotta say: Banning outdoor patio smoking while turning a blind eye to rampant public use of narcotics – for all the families and children to witness – is peak SF brain rot.

@m_saharia Research is not 9 to 5. It’s intermittent and unstructured at times

Yesterday, I was giving an intro talk to our dept's new PhD students. Technical things aside, my number 1 suggestion has remained the same over the years: Treat your PhD like a job. - Avoid 1.5h lunch and three tea breaks. - Avoid gossiping and loitering at work. - Lab at 9 am and leave at 6 pm. Being productive till 11 pm in the lab is a lie people till themselves when their day starts at 1 PM. Everything worth doing can be done with high intensity focus during work hours. And having fun in life is the secret to being productive in a marathon.

@noisyb0y1 Only tools use Claude. Codex and Hermes is the way

A 22-YEAR-OLD FROM BEIJING BUILT A SECOND BRAIN WITH CLAUDE, MAKES $14,000/MONTH AND HIS AI REMEMBERS EVERY DECISION HE'S EVER MADE Most people open ChatGPT, ask a question and start from zero next session, re-explaining the same context every single time and losing 15–40 minutes per session doing it. He built infrastructure where Claude reads, writes and connects everything in Obsidian automatically so it never resets and compounds with every conversation. This morning he needed a client strategy from 3 months ago. Claude pulled it up instantly and drafted a full email reply in under 20 minutes using context it had been building for months across 6,000+ sessions. Every session adds to the last one and the system now saves him 2–3 hours every single day - 30–90 minutes from automated morning briefing alone, before he even opens his laptop. repo: github.com/danielmiessler… The repo is free. The advantage it creates is not.

@AttorneyF_ Islam is the greatest false religion of all time

God made it so easy to spot the falsehood that this religion is, so so so easy.

@randallsquared @IterIntellectus Who steers and understands the AI? Someone basically needs to be accountable and able to explain what the AI is doing.

@ETDataDoc @IterIntellectus What happens when, for anything that needs doing, it's cheaper to create a new AI thread and task it than to hire a human?

i don't get people who say "there will be new jobs with AGI" like how? if AI and robots are truly better than humans at every job we have today, how is it possible for humans to still be competitive? "but every time new tech arrives, new jobs pop up" sure, for the AIs maybe, you don't see horses being hired for transport anymore. if any job were to emerge in the post-AGI era, definitionally AGI would be able to do it better. any company that could be founded would be founded by the AGI before you got there. if it needs dexterity, a humanoid robot already has it. i do not understand how people building cars can tell you with a straight face that there will still be an economy for carriage riders.

@rentbotsTX We need robot stewardesses

We just got robots banned from Southwest Airlines. You’re welcome 🫡 Yesterday we flew our humanoid robot Stewie from Las Vegas to Dallas on Southwest — something we (and others) have tried and failed multiple times because batteries are always the issue. This time we cracked it. Custom lithium pack, spec’d just under the legal limit. Stewie boarded, buckled up, and flew like a completely normal passenger. This morning a Southwest employee leaks us the internal training they just pushed to EVERY flight attendant companywide. Mandatory. Urgent. With a photo of Stewie on the plane as the example of what to look out for. We didn’t break a single FAA rule. Not one. They just weren’t ready for us. Robophobic? Arguably. The robots are traveling whether the airlines are ready or not. 🤖✈️

@kimmonismus I legitimately would buy a phone with an OS that basically solved this problem for control of all of my apps

I think this is bigger than it sounds at first glance. Thinking Machines hasn't just unveiled "ChatGPT, but better." Instead, they've introduced something that addresses a much deeper issue: the very way we interact with AI. So far, AI often feels like email with very clever replies. I say something. Then the model waits. Then it replies. Then I wait. Thinking Machines' new Interaction Model attempts to break down precisely this barrier. It can simultaneously listen, see, speak, interrupt, react, think in the background, and use tools. Not as a cobbled-together pipeline of speech-to-text, turn detection, and agent hacks, but as a native model capability! Good collaboration doesn't happen because someone gives a perfect answer in the end. It happens because someone is present in the moment. If this works, AI shifts from "prompt in, answer out" to something that feels more like collaborative work. A model that notices when you hesitate. That jumps in when it sees something. That anticipates your next move while you speak. That not only gets smarter, but also better at maintaining a flow of conversation with people. ngl really impressed by their examples.

@karpathy Bro Karpathy is the intellectual giant pushing this fields imagination forward

This works really well btw, at the end of your query ask your LLM to "structure your response as HTML", then view the generated file in your browser. I've also had some success asking the LLM to present its output as slideshows, etc. More generally, imo audio is the human-preferred input to AIs but vision (images/animations/video) is the preferred output from them. Around a ~third of our brains are a massively parallel processor dedicated to vision, it is the 10-lane superhighway of information into brain. As AI improves, I think we'll see a progression that takes advantage: 1) raw text (hard/effortful to read) 2) markdown (bold, italic, headings, tables, a bit easier on the eyes) <-- current default 3) HTML (still procedural with underlying code, but a lot more flexibility on the graphics, layout, even interactivity) <-- early but forming new good default ...4,5,6,... n) interactive neural videos/simulations Imo the extrapolation (though the technology doesn't exist just yet) ends in some kind of interactive videos generated directly by a diffusion neural net. Many open questions as to how exact/procedural "Software 1.0" artifacts (e.g. interactive simulations) may be woven together with neural artifacts (diffusion grids), but generally something in the direction of the recently viral x.com/zan2434/status… There are also improvements necessary and pending at the input. Audio nor text nor video alone are not enough, e.g. I feel a need to point/gesture to things on the screen, similar to all the things you would do with a person physically next to you and your computer screen. TLDR The input/output mind meld between humans and AIs is ongoing and there is a lot of work to do and significant progress to be made, way before jumping all the way into neuralink-esque BCIs and all that. For what's worth exploring at the current stage, hot tip try ask for HTML.

@alonzuman @garrytan The future of entertainment is on demand generation of media and video games

Some personal news: I’ve started a company. It’s called Pops, and I’m excited to share that it’s backed by YC and part of the P26 batch. Pops is what I believe to be the future of entertainment: AI-native, playful, social experiences and games anyone can create, remix, and play with their friends. Short-form software. Huge thanks to @Suhail for being an early believer and supporter, and thank you to @garrytan, @greybaker and the YC community for your incredible guidance, support and belief in me — I'm just getting started

@sterlingcrispin @garrytan @_KarenHao Journalists are a fat joke. Time to automate those biased liars

The source of the myth is @_KarenHao she wrote a book called Empire of AI claiming a data center would use over 1,000x more water than a nearby city because she misunderstood the units of measurement. It made international headlines and the mistake / lie has been unstoppable.

@oliverhenry I don’t care if normies are still behind on this. I’m building like a maniac these days

I have still yet to meet someone actually using hermes. Whenever I mention it, i just get bots in my replies. Openclaw 4 lyf

@uncertainsys Yes, our minds have now become the bottleneck

Learning takes far too much energy from humans and in the age of AI, this is rapidly becoming one of humanity’s greatest risks. As AI grows exponentially smarter, we desperately need smarter humans to invent, steer, and fully realize its potential. Yet the opposite is happening.

@TheNorfolkLion It wouldn’t be crazy if the universe God created is one big spiritual filter. Believers are the thing he’s filtering for and nonbelievers, well I’m not sure what happens to y’all. We have some ideas.

As some of you who have known me for a while here know, I’m an atheist. However, I’ve also said that I wish I could believe. I was christened as a child and used to believe. As I’ve got older, I’ve become adamant that there’s no such thing as God. But over the last few years, I’ve been softening and trying to believe again. I’ve asked for so-called enlightenment, but nothing has happened. I read the Bible when I was a child but not since, so I’ve decided to start reading it again. It’s now my nightly ritual before bed. Please send prayers 🙏🏻 let’s hope some enlightenment or something meaningful happens. If God or Jesus don’t come to me after that, I’ll have to carry on being an atheist/realist.

@realBigBrainAI Emergent phenomena don’t really prove reductionism. You got this complex behavior but you still don’t know why

Stephen Wolfram, founder of Wolfram Research, explains how LLMs are quietly dismantling our deepest assumptions about consciousness: He argues that large language models have done something philosophy and neuroscience couldn't: "In terms of consciousness, I have to say, the idea that there's sort of something magic that goes beyond physics that leads to sort of conscious behavior, I kind of think that LLMs kind of put the final nail in that coffin." His reasoning is that LLMs keep doing things people assumed they couldn't: "There were all these things where it's like, oh, maybe it can't do this, but actually it does. And it's just an artificial neural net." Wolfram then challenges a core assumption about conscious experience: the feeling that we are a single, continuous self moving through time. "I think our notion of consciousness is a lot related to the fact that we believe in the single thread of experience that we have. It's not obvious that we should have a persistent thread of experience." He points out that physics doesn't actually support this intuition: "In our models of physics, we're made of different atoms of space at every successive moment of time. So the fact that we have this belief that we are somehow persistent, we have this thread of experience that extends through time, is not obvious." Then Wolfram offers a striking origin story for consciousness itself. @stephen_wolfram suggests it traces back to a simple evolutionary pressure: the moment animals first needed to move. "I kind of realized that probably when animals first existed in the history of life on Earth, that's when we started needing brains. If you're a thing that doesn't have to move around, the different parts of you can be doing different kinds of things. If you're an animal, then one thing you have to do is decide, are you going to go left or are you going to go right?" That single binary choice, he argues, may be the seed of everything we now call awareness: "I kind of think it's a little disappointing to feel that this whole wanted thing that ends up being what we think of as consciousness might have originated in just that very simple need to decide if you are an animal that can move. You have to take all that sensory input and you have to make a definitive decision about do you go this way or that way." The takeaway is unsettling but clarifying. If LLMs can produce complex behavior from simple rules, then consciousness may not be a mystical add-on to physics. It may just be what happens when a layered enough system has to make a decision.

@uncertainsys Yea this is interesting. You’re trying to find a way to maybe compete on eureka moments, which is cool. I think a lot of us are trying to find aha moments all the time.

U can't fake a genuine insight aha moment. I'm thinking participants would have to record their working sessions or smith but usually u can easily see when someone has captured something genuinely new to them. It happens to me sometimes on stream and it's a reaction u can't control nor fake imo. The feeling when smith clicks u know

I'm soooo behind on my Youtube Channel. What are you guys currently learning btw? Weekend is here and I am hungry to learn. I have also been thinking about creating a concept similar to a hackathon but purely for learning. Essentially its a learning marathon? Maybe the goal is to distill some sort of insight and then see who's learned the most obscure thing about a topic? how would you call such event? def not a hackathon but the event we are organizing at ETH Zürich is the spark of something new