Suresh

@danielhanchen would like to see fp8 kv behavior at long context lengths

More NVFP4 dynamic quants! We made them for all Gemma-4 sizes (E2B, E4B, 12B, 26B-A4B, 31B) - they're all W4A4 + FP8 KV cache calibrated + W8A8 for attention / important layers. We also made Qwen3.5-122B-A10B and GLM-4.7-Flash NVFP4 ones as well - 397B and others will come soon!

@kimmonismus k2.6's popularity means the api gets tested hard as soon as k3 goes live.

Kimi K3 may launch tomorrow, and it follows one of the strongest open-model releases of the year. Lets recall: A Kimi Open Platform page announces a limited-time “K3 launch” promotion beginning July 15, apparently revealing the release early (see down below) K2.6 became popular by solving the problems that usually break coding agents: long sessions, unreliable tool calls and rapidly exploding costs. Moonshot demonstrated autonomous coding runs lasting 12–13 hours and involving thousands of tool calls. Vercel reported a 50%+ gain on its Next.js benchmark. CodeBuddy measured 96.6% tool-call success. It remained open-weight, supported 256K context and native vision, generated full-stack applications and coordinated up to 300 sub-agents. At current OpenRouter rates, it costs roughly seven times less than Claude Opus 4.6. OpenRouter now shows 386B processed tokens for K2.6. So this is the baseline from which K3 is now building. K3 signals a major leap. I expect not only longer long-horizon tasks but also a significant improvement over GLM-5.2. Therefore, tomorrow could be a big day for open source!

@FireworksAI_HQ crumpled thermal receipts are a tougher test for receipt JSON

Our free monthly DevRel webinar series kicks off this Thursday, 10am PST. Fine-tune an open vision model to extract clean, structured JSON from messy receipt images, managed start to finish on Fireworks. Plus recent releases and a fine-tuning primer. fireworks.ai/event/fine-tun…

@sirbayes Near saturation, question-level CIs tell more than another thousand samples.

TIL (thanks to CC) about the "Codex pass@k" closed-form estimator from arxiv.org/abs/2107.03374, for the exact expectation over all C(N,k) subsets, with zero subsampling variance. Bootstrap this over questions, et voila, a proper pass@k curve with CIs! 😍

@joelniklaus the gap between DeepSeek V4 Pro and human SLDS scores would matter here

New blog post is out! I evaluate the state-of-the-art in open-source AI on three hard Swiss legal benchmarks: SLDS: write a legal headnote for a Swiss Federal Supreme Court leading decision. Judged 0-100 by DeepSeek V4 Pro on a 5-rubric prompt. SwiLTra-Bench: legal translation across court decisions (sdst), laws (slt), and press releases (sscprt). Judged 0-100 by gpt-4o-mini against a codebook. We report the mean of the three. LEXam: law-exam open questions (judged 0-100 by DeepSeek R1) plus multiple-choice questions scored by accuracy at 4, 8, and 16 answer options. TL;DR: Choose GLM 5.2 for translation, Nemotron or a DeepSeek V4 variant for summarization and open-ended legal reasoning, and Gemma 4 31B for local deployment. Gemma scores 54.8 overall and leads the MCQ benchmark; it fits on local hardware while every model above it is a much larger MoE served in the cloud.

@TONGYI_SpeechAI online RL only fixes rare reading errors when the reward can see them

Voice cloning models usually trade off between three things: reading the text right, sounding like the target speaker, and staying clear and stable. FlowTTS-GRPO gets a model to do all three at once, using online RL on flow-matching TTS.

@rohanpaul_ai @skyfallai cached behavior can look like learning until the rules move

New Continual Learning benchmark, "Morpheus" from @skyfallai Shows an important area LLMs were never properly tested on. Most AI benchmarks reward strong average scores inside environments whose rules barely change. But Continual learning demands more because deployed agents must detect shifts and revise behavior continuously. Morpheus creates those shifts inside enterprise workflows involving resource allocation and scheduling under drift. Each configuration change tests whether performance reflects fresh learning or recycled pretraining habits. Morpheus asks the harder question: does the model update when rules, rewards, and constraints begin to drift? Morpheus therefore separates continual learning from broad pretraining coverage, which conventional averages often confuse. Overall, the benchmark finds that stable benchmark scores of frontier models concealed a harder truth: top frontier models were not truly adapting. Check out their below graph. Task 1 tests how well the system allocates resources, while Task 2 tests whether it can keep scheduling correctly as conditions change. Task 1 stays fairly stable; Task 2 becomes volatile and sometimes collapses.

@rohanpaul_ai at 1.5tb, bandwidth will matter as much as capacity

Apple is reportedly building an M7 Ultra chip that supports up to 1.5TB of unified memory. That would more than double the memory ceiling on today's M3 Ultra Macs. The push comes from local AI, where large models need vast memory to run. The big deal is that M7 Ultra’s GPU could access the entire 1.5TB pool directly. Unlike desktop DIMMs, Apple’s unified memory is package-integrated and fixed when owners purchase it. An Nvidia Blackwell B200 server GPU carries 180GB HBM3e and reaches roughly 8TB/s bandwidth. 8 B200 GPUs together provide 1.44TB, so if M7 Ultra really happens then it will make it genuinely server-scale. Unified memory lets the CPU, GPU, and Neural Engine share one fast pool. So data moves with lower latency and less power than split PC memory. i.e. Apple’s memory would serve CPU and GPU together, unlike NVIDIA’s dedicated GPU-only HBM. Apple’s shared pool reduces copying between processors and gives the GPU far more capacity. Apple's M3 Ultra already reaches 819GB/s of memory bandwidth by fusing two Max dies. Bigger memory would let you run larger models and datasets entirely on-device. That pulls Apple's top Macs into enterprise workstation and AI server territory. It moves closer to dedicated accelerators like Nvidia's Blackwell, without claiming parity. The catch is supply, because DRAM prices are climbing and parts stay scarce. Apple already pulled its 128GB Mac Studio this year amid those shortages. An a 1.5TB part would need far more of that same costly memory. Apple has not confirmed the M7 Ultra, so its final specs could change. Memory supply, not chip design, decides whether this ships at a sane price. If DRAM stays scarce, only film studios and labs could afford a 1.5TB Mac. For some comparsion, an Nvidia RTX 5090 carries 32GB VRAM, but delivers 1.792TB/s bandwidth for graphics workloads. M3 Ultra reaches 819GB/s, so Apple currently offers more capacity but less bandwidth. --- voice .lapaas.com/apples-m7-ultra-chip-may-support-massive-1-5tb-ram-capacity/

@WangYw251 FID numbers stop being comparable once papers use different reference stats.

A subtle but under-discussed inconsistency in image generation eval: The commonly used ADM reference npz (the FID reference stats everyone downloads for ImageNet) is NOT computed from the ImageNet validation set — it's derived from the training set. So here's the weird asymmetry: - To benchmark an autoencoder's reconstruction, we report rFID / PSNR / SSIM on the 50k validation set. - But to benchmark a diffusion model's generation, we compute FID against training-set statistics. If the reference distribution is the training data, aren't we partly rewarding the generative model for memorizing the samples it was trained on? A model leaning toward memorization would look "better" on this FID, not worse. Is training-set FID just an inherited convention, or is there a principled reason I'm missing?

@fahdmirza memory is where that 4k budget gets tested over time

💥 OpenLumara + Ollama is a whole different kind of AI agent ♠ built from scratch for local models, and it's coming for OpenClaw & Hermes 🔹~4000 token system prompt (others burn 15k+) 🔹Everything is a module: memory, scheduler, shell, all toggleable 🔹Locked-down by default: no shell access, no API key visibility, no curl 🔹Writes its own modules to extend itself, on demand 🔹Runs fully local on modest hardware, pairs with llama.cpp & koboldcpp 🔥 Watch the full video below: 👇 youtu.be/gVKkyNznmxw

@james_y_zou Mixed-release UniProt and PDB records can yield very confident wrong joins.

Bio databases/APIs weren't designed for AI agents🧬 So we expanded #Paperclip to transform common molecular databases—UniProt, PDB, ChEMBL—into a unified agent-native virtual filesystem. Now you can do deep research across 10^6 molecules much more accurately and 10x faster! All integrated with millions of full-text papers also indexed in Paperclip. Free to use gxl.ai/blog/adding-bi…

@so_sthbryan repo-wide search can erase a 2x cost advantage in a monorepo.

Coding agent costs halved on real code. Databricks benchmarked agents on their monorepo: - pi-coding-agent 2x cheaper than Claude Code/Codex - GLM 5.2 on par with Opus 4.8 high The benchmark every team will quote this week. databricks.com/blog/benchmark…

@LuminaXspace at a million tokens, retrieval matters more than context size

🚨 GLM-5.5 could arrive in August Z.ai is reportedly preparing GLM-5.5: • Reportedly targeting an August release • Rumoured to contain more than 1 trillion parameters • Expected to build on GLM-5.2’s 1M-token context window • Likely to focus heavily on long-running coding agents to match other western companies No exact day of release have been announced as of yet. With how well GLM-5.2 performs this model should be incredible. Are you looking forward to GLM-5.5?

@GPTWare uneven expert routing will show up fast in mixed-device p99s

Well i've learned a ton while working on this steady. I am getting very close to serving up a quality GLM 5.2 servable on a range of mixed SM120 and SM121 devices. Probably need another 24 hours for the v1 release. This will be a bit of a game changer for those of us with this hardware mix.

@HuggingPapers an AIME24 problem-type breakdown would tell us more than the headline gain

Direct-OPD: Weak-to-Strong Generalization via Direct On-Policy Distillation ByteDance Seed reuses RL exploration from a small model as a dense implicit reward. Boosts Qwen3-1.7B by +10 points on AIME24 in just 4 hours on 8 A100s.

@JoelDeTeves @spiritbuun the long-context drift is what i'd want to see against f16

The biggest benefit of using @spiritbuun "buun-llama" isn't VBR imo, although it's really cool and i'm still playing with it It's the newly added turbo8 kv codec turbo8 > q8_0 - it's near lossless, almost on par with F16 in my testing, again difference between one shot vs two shot The savings over F16 with almost equal fidelity is worth it alone I've been chatting with Buun also, and you can use VBR with a floor set to t8 so mix of F16 + t8, which is insane Really this is the way to go if you're VRAM constrained and want extremely high quality Get it here 👉 github.com/spiritbuun/buu…

@Xudong07452910 Usually, a wrong repo convention gets chosen on turn two and fails at the end.

很多 Coding Agent 看起来是在最后一步失败的。 但这篇论文发现，真正导致失败的错误，往往很早就发生了，只是很晚才暴露。 论文研究的是 CLI Coding Agent，也就是像 Claude Code、Codex、Gemini CLI 这类在终端里做任务的 Agent。作者基于 Terminal-Bench 收集了 3,843 条执行轨迹，最后筛出 1,794 条有效轨迹，覆盖 3 个 scaffold、7 个 frontier models 和 63,000 多个 execution steps。 它有意思的地方在于，没有只看最终 pass/fail，而是去标注失败在轨迹里如何发生。 结果是在 1,184 条失败轨迹中，关键错误的中位数出现在第 7 步；很多失败到第 12 步已经基本锁死；但第一个可观察失败信号通常到第 16 步才出现。 也就是说，Agent 可能已经走错很久了，但日志看起来还很正常。 论文还发现，最常见的失败原因是 epistemic error，占 57.9%。简单说，就是 Agent 已经有足够信息，却忽略、误读，或者基于错误假设继续推进。 这很像我们平时用 Coding Agent 的体感：它不一定不会写代码，但会很早误解环境、目录、需求或报错，然后沿着一个看似合理的方向一路走下去。 这篇论文确实对 AI coding evaluation 很有启发。 只看最后任务成没成，其实不够。更有价值的是看清失败什么时候开始、什么时候变得难以挽回，以及系统有没有机会在中途把它拉回来。 强的 Coding Agent，可能不会从不犯错，而是能更早发现自己正在走错路。 📎 arxiv: arxiv.org/abs/2607.09510

@fahdmirza ram requirements will make or break the no-gpu claim

💥 Colibri 744B is HERE — and it runs with NO GPU ♠ a frontier-scale model that flips the whole "you need a datacenter" assumption on its head 🔹 744B parameters running without dedicated GPU hardware 🔹 Proves massive models can go leaner & more accessible than anyone expected 🔹 Lowers the barrier for devs, researchers & small teams to run serious AI 🔹 Built for those who want frontier power without frontier infrastructure costs 🔥 Watch the full breakdown below 👇 youtu.be/jxML3S5C-8Y

@ModelScope2022 dense tables are the first thing i'd test at 0.8b

OvisOCR2 just released on ModelScope, an exclusive 0.8B end-to-end OCR model for page-level document parsing. 🚀 License: Apache 2.0. 🤖 modelscope.ai/models/ATH-Maa… 🏆 OmniDocBench v1.6 SOTA: 96.58 overall, the first end-to-end model to rank #1 on a leaderboard previously led by pipeline systems. 📄 PureDocBench leader: highest Avg3 score at 75.06, showing strong page-level document parsing beyond a single benchmark. 🧩 Structured output: generates natural-reading-order Markdown, formats formulas as LaTeX, tables as HTML, and visual regions as image tags with bbox coordinates. ⚙️ Compact deployment: post-trained from Qwen3.5-0.8B with SFT, RL, and OPD, with vLLM inference support.

@GuanyangW i'd guess the routine KTV lemmas took longer than the main proof