stableAPY.hl

spent 3 days trying to fine-tune a Qwen 3.5 2B into a terminal assistant and here's what happened I wanted some sort of small LLM living in my terminal. The idea was to have a local macOS assistant that takes natural language and outputs shell commands for this qwen 3.5 2B 8bit seemed the best fit on my m1 pro, 32gb ram. low RAM usage while keeping decent intelligence to evaluate the fine tuning I use an automated agentic benchmark this 60 real world queries run 0 raw model scored 202/300 (67%) on 60 real-world queries. not great but worked for basic stuff and this validated that this was possible to push Qwen3.5 2B run 1 first LoRA pass, 613 training examples, 500 iters. score: 197/300, it got worse runs 2-5 added QA examples, deduped, fixed edge cases. score: 240/300, slow gains run 6 last night I left Opus 4.6 with a set of instructions in order to fine tune this model this morning: 783 corrective examples generated from benchmark failures, a new fine tuning with 600 iters, batch_size 1 score: 274/300. 91% success rate ain't bad for such a small model results on my 60-query benchmark: 67% to 91% thanks to fine tuning, sounds great right? then i tested on queries NOT in the training set, to make sure I was not overfitting and result was 100% The fine-tune wins: no more infinite loops, proper macOS commands, clean output, even tho the base model sometimes picks smarter commands, idk why fine-tuning a 2B model doesn't teach it to reason, it teaches it to pattern-match your dataset and in my case pattern-matching is exactly what I needed, the base model outputs garbage loops and linux commands on macOS and fine-tununing mostly fixed 783 examples is enough to fix format, but not enough to teach judgment, so I'm going further what i'm doing next: scaling to 1500+ diverse examples generated by sonnet 4.6 subagents, with dedup and quality filtering it was fun to watch one AI teaching another AI to use a terminal where I'm at: score: from 202 to 274/300 intent matching: 22% to 78% ~20 to 30 min per training run on my M1 pro. finally, I think the real enemy isn't overfit, it's that 2B params is a really tiny brain, this way every pattern you teach it seems to erases another one Opus tells me that 783 examples is enough so I'm scaling to 1500+ diverse examples generated by sonnet 4.6 subagents, with dedup and quality filtering 3 days ago I knew nothing about fine tuning, model training and was not even aware I could do that locally on my macbook Idk if'll ever reach my goal and if I'm not just overfitting the fuck out of the model, but at least it's pretty fun to do and I'm learning tonnes of stuff

this second fine tuning was meh generated a dataset with 30/70 real interactions/synthetic data, let's see if I can have better results this way pretty fun to fine tune qwen 3.5 2B locally on my M1 Pro 32gb of ram, the process eats all my ressources tho

Jensen Huang is loving the new Dell Pro Max with GB300 at NVIDIA GTC.💙 They asked me to sign it, but I already did 😉

round 2 this time i'm fine tuning it with a real dataset, not some synthetic slop