Ramp Labs

Introducing PorTAL: Portable Task Adapters for LLMs. A novel recipe to cheaply port fine-tuning between models. It matches per task LoRA accuracy at half the cost, lowering the switching overhead of adapting tasks across LLMs. At Ramp, every new model release used to mean retraining our fine-tunes from scratch. PorTAL learns the task once, then efficiently refits it onto any new base model, even across model families.

See how Grok stacks up against the rest here → #head-to-head" target="_blank" rel="nofollow noopener">labs.ramp.com/swebench#head-…

Grok 4.5 launched today. It displays frontier-level shell discipline, chaining Bash commands and filtering their outputs more frequently than any other model tested.

For full write-up, baselines, and ablations: x.com/RampLabs/statu…

In our experiments we train on Qwen3-1.7B and 4B, then calibrate on Qwen3-8B and Gemma-3-4B. Because the task latent and core are reused, each new model needs only a quick converter refit. This matches per task LoRA accuracy in both settings, using half the data and half the cost. The task learning is amortized up front, so porting to unseen models stays cheap.

Introducing PorTAL: Portable Task Adapters for LLMs. A novel recipe to cheaply port fine-tuning between models. It matches per task LoRA accuracy at half the cost, lowering the switching overhead of adapting tasks across LLMs. At Ramp, every new model release used to mean retraining our fine-tunes from scratch. PorTAL learns the task once, then efficiently refits it onto any new base model, even across model families.

Explore the full results → #h2h-claude-sonnet-5-medium-vs-claude-sonnet-4-6-medium" target="_blank" rel="nofollow noopener">labs.ramp.com/swebench#h2h-c…

We ran Sonnet 5 in Ramp SWE-Bench, and observed that compared to its predecessor it: - Runs a high volume of thorough tests - Spends more time in the harness - Costs more and outputs more tokens

GLM 5.2 displays high operating discipline, particularly with its test-driven development style and low tool error rate, despite a fat right tail of longer runs. Compare any two models here → #head-to-head" target="_blank" rel="nofollow noopener">labs.ramp.com/swebench#head-…

New open source releases have moved the Pareto frontier on Ramp SWE-Bench. Kimi K2.7 Code and GLM 5.2 perform stronger than their predecessors at a higher cost, within reach of closed source models.

When measuring effectiveness versus cost, the frontier presents as a tradeoff rather than a single winner. Read our methodology and explore the results below: labs.ramp.com/swebench

Public benchmarks saturate quickly and inevitably leak into training data, with none quite resembling the work our engineers do every day. Building our own benchmark has allowed us to evaluate models within our own financial software ecosystem. We compared models side by side and unearthed their behavioral differences. Head to head breakdowns available here: labs.ramp.com/swebench

Today we’re releasing Ramp SWE-Bench: a private, production-grounded coding benchmark created from real engineering problems we've faced at Ramp.

We built this to earn trust from Ramp customers, who rely on us for their cards, expenses, and payments. If you have a background coding agent, you can build a similar scan for your customers. Full article: x.com/RampLabs/statu…

The scan pipeline is model-agnostic, and does not require a frontier model to drive it. We evaluated several models against our confirmed vulnerabilities, and found that cheaper open-weight models still surface high-severity issues.

We deployed 10,000 background agents to security-scan our codebase. The system is simple, scales with compute, and runs on publicly available models. From the scan, we fixed several high-severity vulnerabilities.

We built a synthetic RL environment with 14 finance task types, gave the model 3 tools and 15 turns, and let it learn how to navigate workbooks on its own. Information retrieval was a huge bottleneck for our spreadsheet agent, fast ask helped solve this. Full writeup: x.com/RampLabs/statu…

This was a good fit for RL because spreadsheet retrieval is repeated often, latency sensitive, and has clean feedback. The model either returns the right cent amount, date, invoice ID, yes/no, or row reference, or it does not. That let us optimize the retrieval policy directly with deterministic rewards.

We partnered with @PrimeIntellect to build Fast Ask, a small RL-trained subagent that helps our Sheets agent find answers in spreadsheets. It scores +4% over Opus on exact match accuracy at Haiku latency.