Pathway (www.pathway.com)

277 posts

Pathway (www.pathway.com) banner
Pathway (www.pathway.com)

Pathway (www.pathway.com)

@pathway_com

Pathway builds the first post-transformer frontier model that solves AI’s fundamental memory problem.

Palo Alto, CA Katılım Haziran 2011
45 Takip Edilen938 Takipçiler
Zuzanna Stamirowska
Zuzanna Stamirowska@zuzanna_pathway·
The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. @lukaszkaiser came to defend the Transformer. @adrian_pathway, @YesThisIsLion, and @mlech26l made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.
English
17
23
210
1.4M
Zuzanna Stamirowska
Zuzanna Stamirowska@zuzanna_pathway·
Pathway and AWS use the term "Sticky Inference" to refer to the part of the AI stack where context compounds. It shows up in the use cases tied to your proprietary enterprise data, where a clear moat emerges as the model keeps learning from the business it serves. @AWSstartups just published a blog on long-horizon reasoning and continual learning for enterprises, and how Pathway's Dragon Hatchling (BDH) is delivering on both. The article covers six use cases across healthcare, financial services, retail and more, showing how the Post-Transformer architecture moves from research to production AI workflows.
Zuzanna Stamirowska tweet mediaZuzanna Stamirowska tweet mediaZuzanna Stamirowska tweet mediaZuzanna Stamirowska tweet media
English
2
2
10
295
Pathway (www.pathway.com)
Pathway (www.pathway.com)@pathway_com·
@probnstat Appreciate the writeup, @probnstat! The 80 minutes flew by because every one of those open questions deserves its own deep dive. Well, Post-Transformer era is here, and we'll have lots of opportunities in the near future.
English
0
0
4
67
Pathway (www.pathway.com) retweetledi
Probability and Statistics
Probability and Statistics@probnstat·
One deep learning debate every AI researcher should care about: Transformers vs Post Transformers. At the surface, it sounds like an architecture fight. Mathematically, it is about scaling laws, memory, online learning in frontier models, and hardware limits. That is what made the recent debate interesting. It featured @lukaszkaiser, @adrian_pathway, @YesThisIsLion, and @mlech26l, hosted by @zuzanna_pathway. Transformers won the last era because multi head self attention scales empirically and fits the hardware ecosystem extremely well. But the next bottleneck may be different. Full self attention has O(n²) compute pressure with sequence length. Transformer LLMs do not natively have persistent long-term memory. RAG retrieves. Longer context conditions. Neither necessarily forms new reasoning patterns inside the model. That is why continual learning is becoming central, recently covered by @a16z. The open questions: – How can models learn after deployment without catastrophic forgetting? – How can long term memory become part of the architecture? – How can models reason over longer horizons without paying infinite context costs? – How can hardware and AI architectures co-evolve more efficiently? – And, are we chasing the right benchmarks with these goals in mind? These questions were tackled head on, with counters from @lukaszkaiser, Transformer co-inventor and core contributor to ChatGPT and GPT models. The image below summarizes some notes from the 80 minute debate.
Probability and Statistics tweet media
English
5
24
75
5.5K
Pathway (www.pathway.com) retweetledi
Sakana AI
Sakana AI@SakanaAILabs·
先日サンフランシスコで開催された討論会「Transformers vs Post-Transformers」に、Sakana AIの共同創業者兼CTOであるLlion Jones @YesThisIsLion が登壇しました。 本イベントは、現在のAI界を牽引するアーキテクチャ「トランスフォーマー」について、論文共著者を含む4人が、トランスフォーマー 支持と、継続学習や潜在空間での推論を武器にその次を見据える「ポスト・トランスフォーマー」の支持に分かれ、これからのAIの未来をどちらが形作るのかを深く議論する場となりました。 その中でLlionは、トランスフォーマーの原論文の共著者でありながら、現在のトランスフォーマーの有用性は十分に認めつつも、あえてポスト・トランスフォーマー側に立ち、その先のアーキテクチャの可能性を論じる役割を担いました。 Llionは、現在のトランスフォーマーの成功は構造そのものによるものではなく、並列処理に優れたハードウェア（GPU/TPU）に適応できたことによる「計算資源の力技」による側面が大きいと分析。それと並行して全く異なる前提に立つアーキテクチャを探る重要性を提起しました。 さらに、今後の研究コミュニティに対して、既存のベンチマークや現在のハードウェアの制約から解放されるべきだと提唱。「次の革新的なアーキテクチャは、初期段階ではトランスフォーマーより遅く、精度も劣るかもしれない。しかし、それを恐れずに全く異なる前提のシステムを探求すべきだ」と、研究姿勢そのものの変革を訴えました。 Sakana AIはトランスフォーマーをベースとした研究開発と並行して、次世代アーキテクチャの探求にも研究にも取り組んでおり、Llion自身が関わっている、生物学的な脳に倣った新アーキテクチャであるContinous Thought Machine（CTM）などはその一例です。 刺激的な議論の場を提供してくださった主催者の皆様、そして登壇者の皆様に心より感謝申し上げます。 当日の討論会の様子は、こちらからご覧いただけます： x.com/zuzanna_pathwa… 🐟 @zuzanna_pathway
Sakana AI tweet media
Zuzanna Stamirowska@zuzanna_pathway

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. @lukaszkaiser came to defend the Transformer. @adrian_pathway, @YesThisIsLion, and @mlech26l made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.

日本語
4
28
183
31.2K
Pathway (www.pathway.com) retweetledi
Llion Jones
Llion Jones@YesThisIsLion·
This was so much fun!!
Zuzanna Stamirowska@zuzanna_pathway

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. @lukaszkaiser came to defend the Transformer. @adrian_pathway, @YesThisIsLion, and @mlech26l made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.

English
1
4
8
1.1K
Pathway (www.pathway.com)
Pathway (www.pathway.com)@pathway_com·
The conversation that anchored last week's debate is now public. Not blog posts. Not Twitter threads. Four researchers who wrote the foundational papers, making the case for what shapes the Post-Transformer era. 📍Transformer vs Post-Transformer: The Deciding Round | May 2026 | San Francisco
Zuzanna Stamirowska@zuzanna_pathway

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. @lukaszkaiser came to defend the Transformer. @adrian_pathway, @YesThisIsLion, and @mlech26l made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.

English
1
2
14
406.2K
Pathway (www.pathway.com) retweetledi
AWS Startups
AWS Startups@AWSstartups·
Transformers unlocked massive productivity gains for startups and enterprises alike. But bigger models and more compute aren't solving the 95% failure rate in enterprise AI. The problem? An architecture with no memory. Pathways BDH is rethinking the foundation — building a post-transformer approach to enterprise AI on AWS that learns and adapts over time.
English
1
5
11
1.3K
Pathway (www.pathway.com)
Pathway (www.pathway.com)@pathway_com·
What comes after the Transformer? Zuzanna Stamirowska puts the debate out in the open, with the very inventors of Transformer and Post-Transformer architectures! Watch the 5-minute highlights. Follow @zuzanna_pathway and hit the bell, full fight drops tomorrow.
Zuzanna Stamirowska@zuzanna_pathway

Transformer vs Post-Transformer: The 5-minute KO compilation is live now. 🥊 @lukaszkaiser (co-invented Transformer & co-created ChatGPT) @adrian_pathway (invented BDH and is CSO of Pathway) @mlech26l (co-invented LNNs & is CTO of Liquid AI) @YesThisIsLion (co-invented Transformer with Łukasz, now CTO of Sakana AI) Moderated by @dexhorthy (CEO, HumanLayer) and me. Full debate drops soon. Turn on notifications to catch the complete fight. This is the ultimate source of truth on the subject.

English
0
3
6
20K
Zuzanna Stamirowska
Zuzanna Stamirowska@zuzanna_pathway·
Transformer vs Post-Transformer: The 5-minute KO compilation is live now. 🥊 @lukaszkaiser (co-invented Transformer & co-created ChatGPT) @adrian_pathway (invented BDH and is CSO of Pathway) @mlech26l (co-invented LNNs & is CTO of Liquid AI) @YesThisIsLion (co-invented Transformer with Łukasz, now CTO of Sakana AI) Moderated by @dexhorthy (CEO, HumanLayer) and me. Full debate drops soon. Turn on notifications to catch the complete fight. This is the ultimate source of truth on the subject.
English
10
11
37
22.7K
Zuzanna Stamirowska
Zuzanna Stamirowska@zuzanna_pathway·
The post-transformer era is here. We’ll shape it together. Heavens! What a fight!🥊 Last night in San Francisco we brought four of the inventors building today’s and tomorrow’s AI architectures into the ring for the deciding round. A year ago, when we said the post-transformer era was coming, most people saw it as just a research concept. Yesterday the room was packed with people who flew in because they felt the shift is real. Research peers from OpenAI, DeepMind, Anthropic, xAI, and NVIDIA. Leaders scaling AI at the largest banks (Goldman, BlackRock, Visa, First Citizens, Merck) and internet companies (Google, Meta, Apple, Microsoft, LinkedIn, Salesforce, Walmart, Waymo). AI investors and diplomats. And founders of other deeptechs in the space. Beyond the stage, what stood out was the willingness to question defaults, surface real disagreements, and keep pushing toward better answers. For this field, and for the people who will build on what we make. Thank you to the fighters who brought that energy: @adrian_pathway (Pathway), @lukaszkaiser (OpenAI), @mlech26l (Liquid AI), @YesThisIsLion (Sakana AI). You did give me a "good clean fight for the AI Champion Title" 🤣. And to @dexhorthy for being my co-moderator that this electric ring needed. Big thanks to our friends in the Bay for spreading the word. I’ll share more over the coming days — there's a lot worth pulling out of those few hours. 📍Transformer vs Post-Transformer: The Deciding Round
Zuzanna Stamirowska tweet media
English
4
9
27
2.4K
Viviana Márquez
Viviana Márquez@vivmarquez·
I never imagined I’d see the creators of Transformers (yes, the "T" in ChatGPT) debating inside a LITERAL boxing ring!!! 🥊 Gotta love living in SF! 🌁 Last week’s battle: Transformers vs. Post-Transformers. My takeaway: Transformers are still incredibly powerful and won’t disappear anytime soon. But as AI hits the limits of pure scaling, we’ll need new architectures that adapt better to the world. The future probably isn’t one it the other. It’s whatever mix of architectures gets the job done. @pathway_com
English
1
0
2
105
Pathway (www.pathway.com) retweetledi
Zuzanna Stamirowska
Zuzanna Stamirowska@zuzanna_pathway·
I largely agree with @YesThisIsLion on this. The biggest mistake right now is expecting the first Post-Transformer models to beat Transformers on day one by delivering massive gains on irrelevant axes.
English
3
4
12
423
Pathway (www.pathway.com)
Pathway (www.pathway.com)@pathway_com·
2 Transformer co-authors. 2 post-transformer inventors. 1 REAL boxing ring. 📍San Francisco
Zuzanna Stamirowska@zuzanna_pathway

Said it would be a real fight. IT WAS. 🥊 @adrian_pathway: “Transformers think in language. They do not think in latent thought.” @mlech26l: “I am convinced that the Transformer will find its own replacement.” @YesThisIsLion: “Lukasz is going to be correct up until that day, and then he is going to be wrong forever.” @lukaszkaiser: “Do not be scared of being 50-times slower!! If you show me a model that is 50-times slower but on a better slope, you win.” Good thing I told them to keep it clean, look at them! 😂 Transformer Vs Post Transformer: Deciding Round, By @pathway_com

Dansk
0
3
4
223

Keşfet

@lukaszkaiser @adrian_pathway @YesThisIsLion @mlech26l @zuzanna_pathway @AWSstartups @probnstat @a16z