Eric Hartford
12K posts
@QuixiAI
We make AI models Dolphin and Samantha BTC 3ENBV6zdwyqieAXzZP2i3EjeZtVwEmAuo4 https://t.co/3ri2GbXrQB https://t.co/zH0F3pTjjY @dphnAI
Making an AI Genie that checks what I'm doing, he's roasting me hard 😭😭
Cartesia’s Sonic-3.5 takes the #1 spot on the Artificial Analysis Speech Arena Leaderboard, surpassing Inworld Realtime TTS 1.5 Max and Google’s Gemini 3.1 Flash TTS Sonic-3.5 is the latest TTS model from @cartesia . It supports 42 languages, including 9 Indian languages, with 500+ voices available out of the box. The model has been highly preferred among voters in the TTS Arena, with its demonstrated naturalness and accurate transcript following. Key takeaways: ➤ Quality: Sonic-3.5 has an Elo score of 1,218 (+16/-16) based on 1,144 arena appearances, placing it ahead of Inworld Realtime TTS 1.5 Max at 1,194 and Gemini 3.1 Flash TTS at 1,209 ➤ Pricing: Sonic-3.5 is priced at $39/1M characters, a premium compared to Gemini 3.1 Flash TTS at $18.3/1M characters, and Inworld Realtime TTS 1.5 Max at $35/1M characters ➤ Speed: 105.5 characters per second, compared to 205 characters per second for Inworld Realtime TTS 1.5 Max and 26.3 characters per second for Gemini 3.1 Flash TTS See more details and listen to samples below 🧵
Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
Most "4-bit training" results come from small models on short token horizons because the format breaks before you can validate it. That's not pretraining — and NVIDIA just drew a clear line between the two. They introduced the first public 4-bit pretraining run at multi-trillion-token scale — a 12B hybrid Mamba-Transformer (Nemotron-Nano-12B-v2-Base architecture) trained on 10 trillion tokens in NVFP4, a microscaling format with 16-element blocks, E4M3 block scales, and an FP32 per-tensor scale, with downstream accuracy closely tracking an FP8 baseline. Here's what's actually interesting: → MMLU-Pro 5-shot: 62.58% (NVFP4) vs 62.62% (FP8). MMLU 76.57 vs 77.36. GSM8K CoT 92.27 vs 89.08. Validation loss within 1% of FP8 in the stable phase → Recipe = selective BF16 (~16% of linear layers) + 16×16 Random Hadamard Transforms on Wgrad inputs + 2D 16×16 weight scaling + stochastic rounding on gradients. Ablations show all four are required → Only linear-layer GEMMs run in NVFP4 — attention, embeddings, normalization, master weights, gradients, and optimizer states stay in BF16/FP32 → On an 8B model, MXFP4 needed 1.36T tokens (+36%) to match NVFP4's loss at 1T tokens Full Analysis: marktechpost.com/2026/05/18/nvi… Paper: arxiv.org/pdf/2509.25149 @NVIDIAAI @ctnzr
MoE vs dense offload on 8GB VRAM MoE offload is 10.8x faster than dense offload on 8GB VRAM. here's the proof. I tested Qwen3.6 35B A3B (MoE, 3B active) vs Qwen3.6 27B (dense, 27B active) on my RTX 4060 Ti 8GB. the numbers: >MoE (-ncmoe 30): 35.4 tok/s >dense (-ngl 20): 3.28 tok/s ratio: 10.8x it gets worse at longer context. at 24K tokens, the gap is 16.7x. MoE has zero context degradation (SSM layers), dense loses -35.4%. why: MoE expert offload keeps the hot path (3B active params) entirely in VRAM. only inactive experts move to CPU when selected. dense layer offload splits every layer across GPU and CPU. every token bounces through PCIe for all 64 layers. the bandwidth bottleneck is fatal. quality is slightly better on dense (5/6 vs 4/6). the 27B model has the best hallucination resistance of all 9 models I tested. if you have 8GB VRAM and a model that doesn't fit: MoE with expert offload, not dense with layer offload.
Node provider rollout has been going well Our pool of inference nodes running Qwen 3.6 35B have generated over 3.2B tokens so far Total inference bandwidth -> 9400 t/s 28x RTX 4090 12x RTX 5090 8x RTX PRO 6000 & many other cards API access coming soon 🐬
Sparse attention mechanisms are finally moving beyond academic benchmarks into production systems, including DeepSeek Sparse Attention, and recently @NousResearch 's Lighthouse Attention. BLASST by NVIDIA, from paper Dynamic Blocked Attention Sparsity via Softmax Thresholding, attempts to sparsify attention in a different way, leveraging a similar rescale factor threshold idea from Flash Attention 4. We expect to see more interesting sparse attention techniques in the future. arxiv.org/abs/2512.12087 (2/4)
