urchade

x.com/i/article/2070…

x.com/i/article/2070…

As promised, we trained an open guardrail model with accuracy matching huge decoder models. Free inference on Pioneer! It’s 16x faster and up to 93x smaller than current SOTA models. It handles multiple tasks, including prompt injection, jailbreak attempts, response harmfulness detection, and refusal classification in a single forward pass. Perfect for use in @nousresearch Hermes Agent and @openclaw. Model weights on @huggingface and free for inference on Pioneer → tinyurl.com/36nrkaw9

x.com/i/article/2054…

We just published a paper on our autonomous fine-tuning agent. The internet found it before we announced it. The paper describes the agent that powers Pioneer, our platform that autonomously fine-tunes small language models end-to-end. Pioneer has two operating modes: cold start (you give it a task description, it handles everything) and production (it retrains deployed models using labeled inference failures). We evaluated cold-start mode across eight benchmarks spanning tasks including reasoning, math, code generation, summarization, classification, and question answering. Fine-tuning performed by the Pioneer Agent improved models by up to +84 percentage points over base. End-to-end runs completed in 8–12 hours at $12–55 per run, demonstrating demonstrating that autonomous fine-tuning can produce high-performing models at minimal cost. A few cold-start results worth noting: ARC-Challenge (Llama 3.2 3B): The base model scored 5.3% because it couldn't follow multiple-choice format. Pioneer Agent brought it to 72.6% over 11 iterations. We also discovered that chain-of-thought supervision via DeepSeek-R1 traces was the decisive breakthrough. HumanEval (Qwen3 8B): When trained on MBPP, the fine-tuned model reached 92.7% pass@1 in just 4 iterations. Interestingly, we found that adding GPT-4.1-generated solutions hurt performance, indicating that external model outputs can dilute the training signal when fine-tuning for basic Python tasks. SMS Spam (GLiNER2): F1 score on SMS spam classification went from 0.159 to 0.997. The final push from 0.98 to near-perfect required adding just 55 targeted examples to the initial dataset. To evaluate production mode, we introduce a novel benchmark: AdaptFT-Bench. AdaptFT-Bench evaluates whether an autonomous agent can fix a deployed model's failures without breaking what already works. It simulates production conditions using synthetic inference logs organized into three stages with increasing noise rates (15% → 25% → 40%), mixing fixable noise with poisonous noise like false premises and label flips. Here are the most notable results from our evaluation of production mode: TriviaQA (Llama 3.2 3B): Pioneer, the Aagent outperformed naive retraining by 43 percentage points by the final stage, the largest gap across all scenarios. GSM8K (Qwen3-8B): Pioneer Agent improved the deployed model from 75.9% to 81.2% as noise accumulated, while naive retraining degraded from 71.6% to 64.7%, demonstrating that the agent gets better precisely where naive approaches get worse. These results demonstrate that the full fine-tuning lifecycle, from task description through production deployment and continuous improvement, can be reliably automated. We also introduce AdaptFT-Bench, a new benchmark for evaluating autonomous model improvement under realistic production conditions. Link to the paper below.

Today, we are launching Pioneer: the world’s first agent for fine-tuning and inferencing SLMs and LLMs. With Pioneer, you can fine-tune and deploy models like Qwen, Gemma, and Llama and achieve state-of-the-art performance in minutes, with a single prompt. Models are continuously optimized on live inference data, meaning that models in production improve over time. Additionally, Pioneer is the only platform in the world to offer fine-tuning for small encoder-based language models including GliNER2, offering frontier-model quality on specific tasks at small-model cost and speed. Start for free at pioneer.ai.

Today, we are launching Pioneer: the world’s first agent for fine-tuning and inferencing SLMs and LLMs. With Pioneer, you can fine-tune and deploy models like Qwen, Gemma, and Llama and achieve state-of-the-art performance in minutes, with a single prompt. Models are continuously optimized on live inference data, meaning that models in production improve over time. Additionally, Pioneer is the only platform in the world to offer fine-tuning for small encoder-based language models including GliNER2, offering frontier-model quality on specific tasks at small-model cost and speed. Start for free at pioneer.ai.

GLiNER2 which is probably 10x cheaper than everything tested here scored 54.79% (1160/2117 matches). Ran on my laptop (no GPU). x.com/kmad/status/20…