.🫟

Finally able to talk about what I've been heads-down on for 6 months at @nvidia 🦀⚡ We just open-sourced cuda-oxide — an experimental rustc backend that lets you write CUDA kernels in pure Rust. No DSLs. No FFI. No source-to-source step. Single source. Short🧵👇

My favorite liang wenfeng story is that when he was in college, a buddy offered him a cofounder position in his drone startup. He turned him down to focus on academics. The drone startup became DJI. You would assume making such a decision would be the defining career moment of someone’s career. That they would regret it forever. But not this guy 😅

tu connectes une clé usb à un android une seule fois elle attrape le cancer

surprised musk didn’t pull up in a blacked out bullet proof model x

Native Windows Is Coming

Hermes Agent is now #1 on the Global @OpenRouter token rankings. While our journey together has just begun, we'd like to take this opportunity to thank our contributors, supporters, and users for all they have done to get us this far.

And they said open-source AI would be worthless!! All of these companies will 5-10x in 1 year

guys we're doing a rebrand of the anomaly stuff so you'll finally stop confusing us with anthropic in the meantime if you're confused remember we're the more handsome but dumber ones

Always read the system prompt before coming to conclusions

linkedin is the real moltbook.

@above_spec i’m assuming tool calling quality is fine at this quant?

Qwen3.6 35B A3B model. 55+ tokens/sec. $300 GPU. No, this isn't a server card. It's an RTX 4060 Ti 8GB. Previously I posted that I 41 t/s on this gpu and that post blew up and went viral. I went back and made it 34% faster. And now the speed doesn't drop with context depth at all. New benchmarks + what changed 🧵

@xsphi @transkatgirl

PlayStation Vita is a great handheld game console. Would be insane if someone were to bring it to iPad and iPhone Vela is coming

who wants autoresearch in opencode desktop?

You don’t pick an Inference Engine You pick a Hardware Strategy Then the Engine follows Inference Engines Breakdown (Cheat Sheet at the bottom) > llama.cpp runs anywhere CPU, GPU, Mac, weird edge boxes best when VRAM is tight and RAM is plenty hybrid offload, GGUF, ultimate portability not built for serious multi-node scale > MLX Apple Silicon weapon unified memory = “fits” bigger models than VRAM would allow but also slower than GPUs clean dev stack (Python/Swift/C++) sits on Metal (and expanding beyond) now supports CUDA + distributed too great for Mac-first workflows, not prod serving > ExLlamaV2 single RTX box go brrr EXL2 quant, fast local inference perfect for 1/2/3/4 GPU(s) setups (4090/3090) not meant for clusters or non-CUDA > ExLlamaV3 same idea, but bigger ambition multi-GPU, MoE, EXL3 quant consumer rigs pretending to be datacenters still CUDA-first, still rough edges depending on model > vLLM default answer for prod serving continuous batching, KV cache magic tensor / pipeline / data parallel runs on CUDA + ROCm (and some CPUs) this is your “serve 100s of users” engine > SGLang vLLM but more systems-brained routing, disaggregation, long-context scaling expert parallel for MoE built for ugly workloads at scale lives on top of CUDA / ROCm clusters this is infra nerd territory > TensorRT-LLM maximum NVIDIA performance FP8/FP4, CUDA graphs, insane throughput multi-node, multi-GPU, fully optimized pure CUDA stack, zero portability (And underneath all of it: Transformers → model architecture layer → CUDA / ROCm / TT-Metal → compute layer) What actually happens under the hood: > Transformers defines the model > CUDA / ROCm executes it > TT-Metal (if you’re insane) lets you write the kernel yourself The Inference Engine is just the orchestrator (simplified) When running LLMs locally, the bottleneck isn’t just “VRAM size” It isn’t even the model It’s: - memory bandwidth (the real limiter) - KV cache (explodes with long context) - interconnect (PCIe vs NVLink vs RDMA) - scheduler quality (batching + engine design) - runtime overhead (activations, graphs, etc) (and your compute stack decides all of this) P.S. Unified Memory is way slower than VRAM Cheat Sheet / Rules of Thumb > laptop / edge / weird hardware → llama.cpp > Mac workflows → MLX > 1–4 RTX GPUs → ExLlamaV2/V3 > general serving → vLLM > complex infra / long context / MoE → SGLang > NVIDIA max performance → TensorRT-LLM

do custom codex usage warnings exist, like warn me when i used 50% of my rolling window

👀Here's @Google running Multi-Token Prediction (MTP) Drafters with Gemma 4. ⚡ Up to 3x faster inference with: 1. Same output quality 2. Runs locally (even on phones) 3. Works in popular open-source tools This is a big win for personal inference and on-device AI! You can do this yourself at home with MTP

I fine-tuned a 6B model under $250 with Codex 5.5 and Deepseek v4 pro. The model beaten GPT-OSS 120B, Qwen 3-32B on all benchmarks. This would've costed be $5,000+ if Deepseek v4 wasn't here. Codex 5.5 pro plan is enough to run for 6-8 hr sprints as an orchestrator and using Deepseek v4 pro model to hand write training material, run tests, submit reports, and iterate. This opened up alot of new opportunities in small language model training space. I'll be posting specific findings on the go. Good times.

Giving away @CommandCodeAI Max subscription to someone at random who follows me and Command. RT. That’s more than 5 billion tokens of DeepSeek v4 pro. In 24hrs. LFG!! Read the eng deep dives below, good for all not just us.