Marq

128!

NVLink installed and showing 14 GB/s bandwidth Next job: getting Qwen3.5 27B running across both 3090s without OOM (Codex is plugging away at it)

I feel sad for people with macbook airs, please live a better life and get a pro .

Sweet Jesus, Mother Mary of Bethlehem God help us all.

It's been running for 6 hours on my first prompt. I have 20+ steering prompts, 25% of them highly detailed. My GPUs have been at 100% utilisation this entire time, my room is 28c boiling hot. I can't describe how crazy it is that this is possible, I am a solo knucklehead imagine what a lab with 1000s of GPUs, and lab equipment can do. Imagine what a nationstate can do. Holy smokes, we are not ready for what is on the horizon.

The core engineering behind @UnslothAI has always been impressive! Instead of relying on PyTorch's default autograd for backpropagation, Unsloth built their own backprop kernels from scratch in OpenAI's Triton language (a Python-based language for writing GPU kernels without needing to write raw CUDA C++). One of the reasons to do this is that the default autograd runs each operation as a separate GPU call, and each call reads and writes data back to global memory before the next one can start. Across dozens of transformer layers, this back-and-forth becomes the real bottleneck. These hand-written kernels fuse operations like QKV projections and rotary position embeddings into single GPU calls, and recompute activations on the fly instead of storing them in memory. This allows Unsloth to deliver >2x faster training with 70% less VRAM without any accuracy loss. The loss curves match standard training runs down to the third decimal because the math is exact, not an approximation. All of these kernel optimizations were already available through Unsloth's Python library. But now Unsloth Studio puts a no-code web UI on top of that same engine, and there's a lot of solid engineering packed into this. > The inference engine has a sandboxed code execution layer where models can run Python and bash, compute results, and verify their answers before responding. This means the model can actually execute and validate code instead of just predicting what the output should look like. The tool calling implementation also has a self-healing mechanism. Failed calls get auto-corrected and retried, which is a practical pattern for agentic workflows. > Unsloth's Python library already had GRPO support (the RL technique behind DeepSeek-R1), and Studio now makes this accessible through the UI. PPO requires running a separate critic model alongside the policy model during training, and that critic is typically as large as the model being trained, effectively doubling the VRAM requirement. GRPO eliminates the critic model entirely by generating multiple completions per prompt and computing advantages from the relative quality within that group. This cuts VRAM by 40-60% compared to PPO. Combined with Unsloth's Triton kernels and QLoRA, training a reasoning model on an RTX 4090 or even a 3090 becomes realistic on hardware that most of us actually have. > In most fine-tuning workflows that I have run, the training step is actually the easy part. Getting raw data into a properly formatted dataset is where the real time goes. Unsloth Studio includes Data Recipes (built on NVIDIA's DataDesigner) that take raw PDFs/CSVs/DOCX files, and transform them into structured synthetic datasets through a visual node-based workflow, replacing the custom parsing scripts entirely. Once training is done, models can be exported directly to GGUF, safetensors, or other formats with automatic LoRA adapter merging into base weights. The whole system runs 100% offline with no telemetry. $ pip install unsloth $ unsloth studio setup $ unsloth studio More details in the post below by Unsloth👇 It's still in beta, but the engineering underneath is solid. For anyone working with open-source models locally, this is one of the more complete tools available right now. ____ Find me → @_avichawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

351. The Philosophy of Nick Land with Vincent Lê Listen now: anchor.fm/hermitix Youtube: youtu.be/6AeeoCYB7Ys --- Patreon: patreon.com/hermitix

It’s fun at the top :)

Learning so much from your book, an invaluable resource.

Funniest thing to do is take Claude codes code and give to regular Claude, it’s always like no this is awful 😂

My startup is working to increase inference speeds.

A thoughtful review of my book on the "eternally relevant" Wittgenstein: prospectmagazine.co.uk/culture/72737/…

Please consider reading my newest article, Semiotic Triage: Overcoming the Type Error Tragedy. In it, I discuss how we might suspend our disbelief long enough to use faith as a generating function for enchantment, going beyond our present epistemic limitations.

you don't understand anon. i'm on a mission to find the collection of best small models that run full context on consumer hardware. because when you can orchestrate your own thinking across physical nodes locally, that's not a tool anymore. that's an extension of your mind. that's exactly where we are headed as a civilization. and most people haven't felt it yet.

I'm working on the coolest project I've ever coded, and probably the hardest, of course this is turning into a startup.

I think my head of research is starting to develop schizophrenia

A mathematical theory for understanding when abstract representations emerge in neural networks arxiv.org/abs/2510.09816

🔥

Hermes Agent really does start out a bit more of a blank slate than OpenClaw and it feels like it’s not as smart out of the box. But as you use it, it starts getting smarter and smarter, and it doesn’t take long, just some patience. By contrast OpenClaw seems to get dumber.