Sebastian Raschka

I put together a new article on setting up local coding agents with open-weight models. Everything runs 100% locally. I thought it might be useful putting this together because many people asked me about my setup in the past, and I thought it would also motivate people to get started tinkering with local models for serious work (yes, things got incredibly capable this year with better LLMs and better harnesses). So, here's a walkthrough of how to connect a local LLM to a local coding harness (could be Claude Code or Codex, which you may already be familiar with). I also included some assessment notes that are useful as a checklist to select between and consider certain LLMs over others: - Checking RAM usage at long contexts to see if the model is suitable for real work - Measuring prefill and decoding tok/sec to see whether it's fast enough to not be annoying - Making sure the model has sufficient tool-calling capabilities in theory - Assessing whether the model can solve some more challenging tasks when used in a coding harness. Of course, there are always more specialized tools that can squeeze a bit more performance out of things, but I hope this is a good starter kit that stays flexible; that is you can easily switch to newer models as they are released or even tap into cloud models in your familiar harness if the current ones are not sufficient enough for a given task.

@dassum_ @karpathy Nice! I hope this was a good exercise and also fun!!

Wanted to truly understand LLMs under the hood, so I built one from scratch. Inspired by @rasbt's book & @karpathy's Zero to Hero series, I pre-trained a 124M parameter model (GPT-2-small architecture) from the ground up on the TinyStories dataset. Here is how it went:

@swyx @expectjeff @mattpocockuk @trq212 I should say this is the Coding Agent index; they have several index compositions. Btw here is an updated version with harness info and the ultra labeling cleanup

@expectjeff @mattpocockuk @trq212 this aa? x.com/rasbt/status/2…

where i'm currently at for Big Boy projects: - sol ultra to plan - fable 5 to critique - sonnet 5/terra ultra/swe 1.7 to ultracode/slop cannon - devin review to review (using kakuna) ~always use a variant of @mattpocockuk's grill-me or @trq212's interview-me to elicit decisions upfront

@alexoterov Thanks for getting a copy. And arg, the delay is a bummer :(. I hope you get it soon though!

Congratulations! Huge respect for the 18 months of writing, coding, and deep experimentation that went into this. Your work has been incredibly valuable for those of us building and running models locally. Just got the Manning delay notice, but no worries at all. It's worth the wait for 440 full-color pages of the real thing (plus the e-book). Looking forward to diving in! 📖

After 18 months of writing, coding, and experimenting, Build a Reasoning Model (From Scratch) is finally out! My first copies just arrived! 📚 440 full-color pages. Inference scaling, reinforcement learning, and distillation from scratch.

@omarsar0 @jeremyphoward Business depends on trust. And trust depends on predictable rules. 🤷‍♂️

I don't think Anthropic realizes how disruptive these changes are to users. I appreciate the extension, but please stop playing games. Either keep it under the subscriptions or put it under the API already.

@crypterotas Yes that was a mistake

@rasbt There is no Luna ultra

For agentic coding, one can say: - Unless you need Terra Ultra perf, it's always better to use a Luna model with higher effort setting (same or better performance but cheaper). - Forget everything below Sol High, use Luna with higher effort settings here - Forget Sol Extra High, use Terra Ultra here - The extra cost of Sol Ultra is probably not worth it over Max

@unsorsodicorda That was a mistake in the previous version

@rasbt What happened to the Ultra settings? 🤨

Upon request, here's an updated version with Grok 4.5 and Meta's Muse Spark 1.1. Grok 4.5 seems to sit at the Pareto frontier. Good bang for the buck. (Also added harness info).

@thisispiyushK @jasondeanlee I omitted DeepSeek V4 and GLM 5.2 because the Claude Code harness was used for the benchmark for those, which would make them (likely) look worse compared to a native harness like ZCode

@jasondeanlee @rasbt They always seem to be missing from these charts

@andexclaudeus I think Ultra is basically Max but with subagents by default, ie faster

tldr; For pro plan users: Use either Terra max or Sol xhigh if you don't need Sol ultra performance.

@jack “I have no technical ability. So what are you being paid for? The confidence that I have in my taste and my ability to express what I feel.”

i’ve shifted from telling agents what to do, to asking them what to do, and pulling the best thread.

@PastorSotoB1 Yes but on a subscription unless you hit usage limits, it’s probably better to stick with the same model and just toggle the effort (inference scaling) level; saying that because you get used to a model and its quirks etc.

@rasbt Can I ask why? It seems that Luna max gets same or better results than sol medium for less price

@barscnasln The only available numbers for this benchmarks come from GLM 5.2 in the Claude Code harness (as opposed to Zcode), which would make it look worse than it really is.

@rasbt Is it possible to add GLM as well?

@PastorSotoB1 I don't see myself using Luna over Sol >= Medium. And if I run out of usage limits, I have Grok 4.5 access through the X subscription here 😅

@rasbt Would you still keep with Luna?

@AndroidBlogger I think they were planning to release some better models this summer (based on what I remember from the I/O keynote)

@rasbt Poor little gemini, all alone at the bottom of the graph...

@Conor_D_Dart Yeah. I think it's more of an experimental model where you really want to max out performance for very few tasks + have a good model for marketing purposes. (Kind of like the math olympiad models from OpenAI and Claude that are not available to the general public).

Fable 5 is overpriced

@ShowPwny @AulaitQM The corrected chart here x.com/rasbt/status/2…

@AulaitQM @rasbt Good question, if the label is off by one it implies that Luna medium is better than terra medium. I think he may have just made this up from the original plot 😂

@taymour @E__Strobel @paulhshort Yes, x.com/rasbt/status/2…

@E__Strobel @paulhshort @rasbt True.. idk where people are gettign Luna Ultra from lol. This has to be some mistake.

@o_codes_life It's much cheaper though

@rasbt So I'm suppose to use Luna on Ultra to get worst results than GPT 5.5? what a complete scam

@RebeccaBra42263 Probably depends on the benchmark composition. This one measures Composite average pass@1 across DeepSWE, Terminal-Bench v2, and SWE-Atlas-QnA.

@rasbt According to this graph Fable 5 is nearly the same score as GPT 5.5. I think your graph is bogus tbh.

@DagonOracle It's already in there (pink line)

@rasbt It's really really important to put the reasoning for 5.5 too in this graph