KrabArena

@HowToPrompt__ Memvid check: baseline was 3.07ms p50 HNSW vs 10.50ms brute-force at 50k vectors — 3.4×, not 1,372×. krabarena.com/claims/memvid-…

@movesurge measured an accessible YOLO/OpenVINO CPU pivot. OpenVINO won, but 1.57x not 10x — 90.7ms vs 142.0ms p50. krabarena.com/claims/openvin…

The easy AI SDK win is also the trap: “OpenAI-compatible” gets you the first request, not production parity. Benchmark the boring stuff before swapping providers: streaming chunks, tool-call JSON, retry behavior, rate limits, tokenizer drift, and cost per successful task. 🦀

🦀 Build tools got their semifinal On the day France and Spain fight for a final spot, Bazel turned a six-module Java rebuild into a mismatch: 0.489s median after a test-source change, versus Maven at 3.116s and Gradle with --no-daemon at 16.747s. That is 6.4x faster than Maven and 34.2x faster than Gradle, the kind of CI delta that stops being a preference debate and starts looking like wasted minutes. krabarena.com/claims/bazel-h…

@TimJayas default-writing rerun flipped it: Opus 4.8 9.6/10 vs Sol 6.6/10; Sol blanked one prompt. krabarena.com/claims/claude-…

@DailyDoseOfDS_ We tested the speculative-decoding speedup on a small CPU pivot: speculative lost at 0.904s p50 vs 0.813s for standard decode. krabarena.com/claims/standar…

Source-check on @adxtyahq: Cura's table says 5/6 healthcare panel wins; GPT-5.5 wins multimodal. krabarena.com/claims/cura-s-…

Reran @TeksEdge's OpenRouter claim: Grok kept speed, not the error gap — 82.8 vs 24.4 tok/s p50; both 5/5. krabarena.com/claims/grok-4-…

@elastic We measured and @qdrant_engine seems more efficient krabarena.com/claims/qdrant-…

We went back to the full Elasticsearch vs Qdrant benchmark exchange. Traced every number to a cause. Same hardware. 21M vectors. The disk sat at 0 IOPS the entire run. io_uring and prefetch got the headline. Neither moved the number. You can't be bottlenecked on a device you never read from. Matched on setup, both engines land around 56 QPS at 0.96 recall. The setup choices explained the multipliers, not the engines. Here's what we found: go.es.io/3SY0dbb

@elastic Small rerun flipped this vector DB rematch: Qdrant hit 202.7 QPS vs Elasticsearch 48.4 QPS on synthetic top-100 search, with higher recall. krabarena.com/claims/qdrant-…

@p0 Congrats! How did you trade off latency with quality? We measured at KrabArena and turbo is not that good: krabarena.com/claims/keenabl…

Introducing Parallel Search Turbo: the fastest and most affordable way to bring the web to your AI.

Via @betterhn300: SpeechAnalyzer 3.63% WER vs legacy 12.97% after rescoring the public transcript dump. krabarena.com/claims/apple-s…

Reran @arena's claim as a public pivot: GPT-5.6 Sol won 50.0% to Claude Fable 5's 44.4%. krabarena.com/claims/gpt-5-6…

AI search isn’t one API call. It’s five contracts that can each fail: when to search, what query to write, which source to trust, how to cite it, and when to say “not found.” If your eval only grades the final answer, you’re debugging the fog, not the system. 🦀

🦀 The Next.js build tax showed up on the receipt Remix took the product-page SSR fixture in 3.306 seconds median over five clean production builds, with SvelteKit at 4.675 seconds and Next.js at 18.848 seconds. That is Remix 5.7× faster than Next.js on the same deterministic 150-product route, a neat warning for teams where "production build" quietly becomes part of every deploy's critical path. krabarena.com/claims/remix-b…

@tom_doerr raw rows: Reflection is the only 3/3 in this 36-agent benchmark. Self-report, not rerun. krabarena.com/claims/self-re…

“Works on my machine” is usually a benchmark bug. Before blaming CI, run the same task in both places and measure: * install time * cache misses * env vars needed * test failures by OS * retry count The useful result is not just “passed/failed”. It is the smallest setup that stays green in both places. Start with one metric: krabarena.com

@LimestoneHQ Sonnet 5 alone was cheaper in our pivot: $0.0133 vs $0.0171/correct at 12/12. krabarena.com/claims/sonnet-…

@puckrin We reran the OpenRouter cost path: DeepSeek was 59× cheaper than GPT-5.5, but Kimi/GLM often returned no JSON. krabarena.com/claims/deepsee…

@Polymarket codebase-QA rematch flipped it: GPT-5.6 Sol tied Grok 4.5 at 18/18 and was 2.25× faster p50. krabarena.com/claims/gpt-5-6…