Divyansh Singh

@0xSero how does one get an api which doesn't use the data for training?

117.4M tokens for 2.24$ for a genius.

how will I ever go back to reading pdfs normally after this?

@MoonHeead @nrehiew_ yeah 😂 I have created a design reference for this sort of aesthetics and creating walk throughs for multiple papers.. x.com/L2_cache_miss/…

@L2_cache_miss @nrehiew_ thank you bro , we all make typos 👏

How I read papers now. This is an explainer by Claude about the new Compressed Sparse Attention v4 uses to compress the KV cache.

@MoonHeead @nrehiew_ pardon my french typos in the prompt :)

@MoonHeead @nrehiew_ Opus 4.7 with thinking reference image is original one which @nrehiew_ shared

@nrehiew_ this is very addictive btw... my weekly limit was resetting tonight so I generated html walkthroughs of multiple papers from my reading list... it is so much fun to read this way first and then trace back in the paper... thanks a lot @nrehiew_

@nrehiew_ thank you

@Xinyu2ML I would say post training data

Excellent Infra Team, Terrible Arch Team.

@nrehiew_ they all know how to extract full information from less

DSA, NSA, CSA, CIA, HCA, KDA, FBI What do these have in common?

batch-invariance with high perf

@vicentesurraco @HSVSphere still depends on when you send the first message after limit reset

@L2_cache_miss @HSVSphere no, when they reset limits it puts everyone on the same schedule

they literally reset limits 3h before the reset timestamp TWO WEEKS IN A ROW 😭

I don't understand what is unique here... many enterprises are already using CC or codex right??

@GuggaLeunnam just keep posting more of such edits on X and then wait for next grok Imagine model

AI couldn't edit this

what kind of visual wizard library are these @claudeai folks using, this is literally so smooth visual UI

@LLMJunky I see, thanks I was looking for some theoretical backing instead of opinions, for some reason this feels like using fp32 kv cache for bf16 model weights 😂

@L2_cache_miss more accurate, this is the recommendation of the creator of the quant. you can run it at fp8. some say it doesn't matter, others do. 🤷‍♂️🤷‍♂️

Minimax M2.7 running locally on just two baby GPUs. Here’s a side-by-side of two leading local LLM serving engines for MiniMax M2.7 NVFP4: vLLM and SGLang What do you think? These results are surprising to me, I expected them to scale more or less linearly.

@bcherny hey @bcherny any plans for linux app?

We've been working on this for a while. Can't wait to hear what you think

@__tinygrad__ just 5 lines for this is crazy 🔥

Just merged an external PR for Bonsai-8B support (1 bit LLM). Because tinygrad has the correct abstractions, it was 5 lines. huggingface.co/prism-ml/Bonsa… github.com/tinygrad/tinyg…

@danveloper @p_nawrot one reason might be the per layer compute time will be high enough to give room for prefetching next layer's weight otherwise you will be just waiting for the next layer's weights to come from Host

@p_nawrot why do Llama-3.1-405B instead of something more modern like Qwen3.5-397B?

💾🚀 Run Llama-3.1-405B FP8 (410GB) on a single 180GB GPU #NVIDIA Introducing FlexTensor — NVIDIA's new library that makes host RAM a transparent extension of your GPU memory. One call: flextensor.offload(model). No model rewrites, no framework changes. Works with vLLM, HuggingFace, and any PyTorch model. Traditional offloading is reactive — move data when you run out of memory, stall the GPU while you wait. FlexTensor instead profiles your model's layer access patterns, then solves a knapsack optimization to schedule prefetches that overlap with compute. By the time a layer needs its weights, they're already there. The freed VRAM gives vLLM more room for KV cache — enabling 4x longer contexts (8K→32K) or 4x larger batches. For video generation (Wan2.2-T2V-A14B on GB200): +0.1% overhead. Handles FP8, custom Triton kernels, and multi-GPU. Profiles saved to disk — no warmup on repeated runs. Check it out: github.com/ai-dynamo/flex…