David Hendrickson

16K posts

David Hendrickson banner
David Hendrickson

David Hendrickson

@TeksEdge

CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering https://t.co/9oqvHuTX5f | 🔔 Follow for AI & Vibe Coding Tips 👇

PNW Katılım Temmuz 2023
539 Takip Edilen8.5K Takipçiler
Sabitlenmiş Tweet
David Hendrickson
David Hendrickson@TeksEdge·
🎗️ "Medium-Sized" LLM Burners Coming Soon! 🔥 This Could Make Local HyperToken Generation a Reality. ⚡️ NVIDIA’s worst nightmare? 😱 ⚙️ Application-Specific Hardware Taalas new PCIe ASIC board would burn the entire medium-sized Qwen 3.5-27B LLM straight into silicon 🤯 (already doing it with small models) Taalos said medium models on ASIC would be available in their lab by Spring '26. 💭Imagine: 🚫 No more loading weights 🚀 ~10,000 Tokens Per Second locally (Llama 3.1 8B already @ 17,000 tps) 💻 Standard PC slot, ultra-low power (10x less) 🔋 🌍 100% offline with no cloud, no GPU farm 💰 Reddit unit cost rumor $300 to $400 🖥️ Imagine HyperToken generation on your desktop. 🤖 AI agents that think at light speed. ⚡️ Are you ready? 👀
David Hendrickson tweet media
English
179
422
2.7K
496.5K
David Hendrickson
David Hendrickson@TeksEdge·
🖥️ A new 128GB option for Local AI is coming. Minisforum MS-03: Panther Lake Local AI Edge Workstation What it is: Compact mini workstation (successor to MS-01) with Intel Core Ultra 9 386H (Panther Lake, 16 cores: 4P+8E+4LP-E, up to ~4.9GHz, 50+ TOPS NPU). ✅ Up to 128GB DDR5-7200 SODIMM (socketed) ✅ PCIe 5.0 x16 slot (x8 electrical) for low-profile dGPU / 10G networking / capture cards ✅ 3x M.2 (one convertible to U.2) + dual 10GbE SFP+ + 10G/2.5G LAN ✅ Dual USB4, Wi-Fi 7, blower cooling Perfect for local AI agents, edge computing, homelabs, Proxmox, or small biz servers. Price: TBD (barebone/configs expected in premium mini-PC range; sign up for $50 launch discount). When: Pre-order / launch happening now — global availability soon
David Hendrickson tweet media
English
4
3
22
3.2K
David Hendrickson
David Hendrickson@TeksEdge·
Here's the latest on DeepSWE. GPT-5.6-sol dominates Fable 5. Where is Muse Spark 1.1 or GPT 4.5?
David Hendrickson tweet media
English
4
1
14
1.4K
David Hendrickson
David Hendrickson@TeksEdge·
Is it me, or is Satya Nadella, Microsoft CEO, advocating Local AI? 🔒 Keep data, prompts, traces & evals inside a trust boundary 🧠 Own memory, feedback & adapted weights 🔄 Avoid model lock-in 🏢 Control your own learning loop Not desktop-only—but clearly sovereign/private AI.
Satya Nadella@satyanadella

x.com/i/article/2076…

English
0
0
3
275
David Hendrickson
David Hendrickson@TeksEdge·
To those unfamiliar, GPT has always been a dog (tired and slow) when it come to frontend coding. In my experience Claude and Gemini have always done a better job without fail. Someone at OpenAI must have noticed and made huge gains in this regard. Congrats to @OpenAI .
David Hendrickson tweet media
Design Arena@Designarena

BREAKING - OFFICIAL RESULTS: GPT-5.6 Sol by @OpenAI is 1st overall on Design Arena with an Elo of 1353. This puts GPT-5.6 Sol above Claude Fable 5 by @AnthropicAI and in the same performance band as GLM 5.2 by @Zai_org on frontend design. This is an 18-position and 60-point Elo leap from GPT-5.5. GPT-5.6 Sol also establishes a new Pareto frontier for preference vs. speed, faster than any model at this performance. Congratulations to the @OpenAI team on the launch!

English
2
2
8
691
David Hendrickson
David Hendrickson@TeksEdge·
This a feat or engineering even if performance is unusable. Fitting such a large model in such a small amount of memory at least deserves applause. Faster secondary storage for memory offloading may help with RAM shortages.
David Hendrickson@TeksEdge

🚀 Holy 💩! Major Local AI Breakthrough! 🧠 744B-parameter GLM-5.2 (1.5 TB total) is now running on just ~25 GB RAM — no discrete GPU required! 👀 Wut!? Must test!! Italian engineer @JustVugg built Colibrì, a pure C inference engine (single ~2.4k line file, zero runtime deps) that: • 🛡️ Keeps the dense core (~10 GB at int4) resident in RAM • 📀 Streams 21,504+ MoE experts from fast NVMe on demand (only ~40B active per token) • ⚡ Supports native MTP speculative decoding + MLA attention Result: Frontier-class model on everyday consumer hardware! 📊 Current speeds: • 25 GB RAM setup → 0.05–0.1 tok/s (disk-bound) • Higher RAM + fast SSD → up to 1+ tok/s (warm) • It's a start... what could you do with a 5090? 💡 Big opportunity: Pair it with Phison aiDAPTIV+ AI SSDs to kill the I/O bottleneck 👉 smarter caching, prefetching & KV offload could make it dramatically faster! This is a huge step toward truly accessible local frontier AI. 🔗 GitHub: JustVugg/colibri

English
4
2
20
2K
David Hendrickson
David Hendrickson@TeksEdge·
We've had Qwen3.6-27B and 35B since April and Qwen3.7 Pro/Max since May. Wonder if Qwen3.7 is coming for medium and small.
English
11
1
52
11K
David Hendrickson
David Hendrickson@TeksEdge·
✨ Local model speedups incoming! 🙏 🪄 Pending PR for llama.cpp will supercharge local AI inference speeds llama.cpp is adding DFlash, a new speculative decoding method that can deliver much higher speedups than current options! 🔄 Current Speculative Decoding Status: ✅ MTP — Already supported (results vary) ✅ EAGLE3 — Recently added 🆕 DFlash — Currently in a pending PR (not merged yet) ⚡ What makes DFlash different? Unlike MTP, DFlash uses a draft model to generate blocks of tokens in one pass. Early tests show 3–4x+ speedups on Qwen3/Qwen3.6 models — often outperforming MTP. 📦 LM Studio Note: LM Studio uses llama.cpp as its backend, so DFlash will eventually be available there too. However, speculative decoding in LM Studio has historically been inconsistent for many users. 🔥 This is another solid step toward faster and more efficient local inference
David Hendrickson tweet media
Georgi Gerganov@ggerganov

llama.cpp recently added DFlash support to its speculative decoding arsenal. Along with MTP, Eagle3 and various ngram-based techniques, the local model performance takes another step up. Special thanks to NVIDIA team and Ruixiang Wang specifically for leading this effort! github.com/ggml-org/llama…

English
1
2
25
2.3K
David Hendrickson
David Hendrickson@TeksEdge·
🎉 Congrats to Grok 4.5 and Anthropic's Fable 5 for moving up the Omniscience benchmark, the only measure of LLM trustworthiness. 📊 AA-Omniscience Index measures knowledge reliability and hallucination resistance (higher = better). Current Top Scores: 🏆 Claude Fable 5 (with fallback): 40 🔹 Gemini 3.1 Pro Preview: 33 🔹 Claude Opus 4.8 (max): 27 🔹 Grok 4.5 (high): 26
David Hendrickson tweet media
English
4
1
32
2.1K
David Hendrickson
David Hendrickson@TeksEdge·
🚀 Holy 💩! Major Local AI Breakthrough! 🧠 744B-parameter GLM-5.2 (1.5 TB total) is now running on just ~25 GB RAM — no discrete GPU required! 👀 Wut!? Must test!! Italian engineer @JustVugg built Colibrì, a pure C inference engine (single ~2.4k line file, zero runtime deps) that: • 🛡️ Keeps the dense core (~10 GB at int4) resident in RAM • 📀 Streams 21,504+ MoE experts from fast NVMe on demand (only ~40B active per token) • ⚡ Supports native MTP speculative decoding + MLA attention Result: Frontier-class model on everyday consumer hardware! 📊 Current speeds: • 25 GB RAM setup → 0.05–0.1 tok/s (disk-bound) • Higher RAM + fast SSD → up to 1+ tok/s (warm) • It's a start... what could you do with a 5090? 💡 Big opportunity: Pair it with Phison aiDAPTIV+ AI SSDs to kill the I/O bottleneck 👉 smarter caching, prefetching & KV offload could make it dramatically faster! This is a huge step toward truly accessible local frontier AI. 🔗 GitHub: JustVugg/colibri
David Hendrickson tweet media
adcker ☄️@adckerX

A 744-billion-parameter AI model running on a regular computer with only 25 GB of RAM—and no GPU. 🤯 Colibrì is a lightweight, open-source inference engine written in pure C that can run GLM-5.2 on consumer hardware. Instead of loading the entire model into memory, it keeps RAM usage low by streaming only the required Mixture-of-Experts components directly from an SSD while generating each token. The impressive part: ⚡ Pure C 📦 Zero runtime dependencies 💾 Around 25 GB of RAM 🧠 744B total parameters, with roughly 40B activated 🚫 No expensive GPU required There is an important trade-off: this is not fast inference, and the quantized model files still require hundreds of gigabytes of SSD storage. But as a technical proof of concept, it is remarkable. It shows that running enormous AI models locally may depend as much on clever memory management as it does on expensive hardware. Tiny engine. Massive model. Very smart engineering. 🐦💻 Repository link below 👇 The model is officially listed as a 744B-parameter MoE with about 40B active

English
23
23
210
22K
David Hendrickson
David Hendrickson@TeksEdge·
Even with all the new model releases and the significant improvement in capabilities the IQ wall of 130 points persists and is a level of intelligence that can't be broken. Neither Fable 5 nor Grok nor Spark 1.1 can beak through.
David Hendrickson tweet media
English
5
2
36
2.5K
David Hendrickson
David Hendrickson@TeksEdge·
July '26 LLM list (updated) Fable 4.6 ❌ GPT-6 (Spud based) ❌ GPT-5.6 Sol Ultra ✅ GPT-5.6 Sol ✅ GPT-5.6 Terra ✅ GPT-5.6 Luna ✅ Meta Muse Spark 1.1 ✅ Meta Watermelon ❌ Gemini 3.5 Pro ❌ Grok 5 ❌ Grok 4.5 ✅ Qwen3.7-30B ❌ 🙏 Kimi K3.0 ❌ Hunyuan3 MoE ❌ Nano Banana 2 Lite ✅ Gemini Omni Flash ✅ Leanstral 1.5 ✅ SWE-1.7 ✅
Indonesia
2
0
33
4.5K