Quesma

Recently we built OTelBench – a benchmark to test how well LLMs handle OpenTelemetry instrumentation. We tested 14 models. The best (Claude Opus 4.5) hit only 29%. These weren't trick questions, just small subset of typical SRE tasks. Link here: quesma.com/blog/introduci…

AI + Ghidra by NSA = reverse-engineering fun I am speaking at @AITinkerers Warsaw, 4th Mar 2026. One of my favorite event series - by and for the creators community. Vibe-resurrecting an old game from binaries 👾 and vibe-hardware-ing a LED backpack 🎒🌈.

Claude can code, but can it read machine code? We gave AI agents access to Ghidra (a decompiler by the NSA) and tasked them with finding hidden backdoors in servers - working solely from binaries, without any access to source code. See our BinaryAudit: quesma.com/blog/introduci…

Great to see the community releasing benchmarks in @harborframework now. These are invaluable resources for collectively building the most useful agents.

I used to cite Gartner, now I quote @GergelyOrosz and his Pragmatic Engineer. Enjoy our new blog post: quesma.com/blog/prompts-s…

Finally, an AI that can draw a map without getting lost. Nano Banana Pro uses tools to create factually correct infographics - and it's a game-changer. quesma.com/blog/nano-bana…

Postmortems are painful to write, especially this one. Sharing my startup Quesma journey so far. quesma.com/blog/database-…

Interesting use case for AWS Lambda that we explored: sandboxing AI-generated code. We tried WebAssembly first but hit the wall. So, we scrapped our experiment for AWS Lambda with Docker containers in an isolated VPC. Full writeup from @pmigdal: awsfundamentals.com/blog/sandboxin…

The security paradox of local LLMs - quesma.com/blog/local-llm… by @jakozaur at @QuesmaOrg If you’re running a local LLM for privacy and security, you need to read this. Our research on gpt-oss-20b (for OpenAI’s Red‑Teaming Challenge) shows they are much more prone to being tricked than frontier models. When attackers prompt them to include vulnerabilities, local models comply with up to 95% success rate. These local models are smaller and less capable of recognizing when someone is trying to trick them. #AISecurity #LLMSecurity #LocalLLM #GenAI #MLOps #ModelRisk #DataPrivacy #AIPrivacy #PromptInjection #AIThreats #AIGovernance #EdgeAI

See the full ranking and every run (logs, commands, binaries), methodology & code: ▶️ compilebench.com 💻 github.com/QuesmaOrg/Comp… 📃 quesma.com/blog/introduci…

Cost-efficiency crown: @OpenAI. Across difficulties, OpenAI models dominate the Pareto frontier of cost. GPT-5-mini (high reasoning) is a great price/perf pick; GPT-4.1 is the fastest with solid wins.

Can AI compile 22-year-old code? We built CompileBench to find out. We know that LLMs can vibe-code or even win IOI, but what about dependency hell or legacy build systems? (image based on XKCD 2347)

Our blog post is second on Hacker News. Enjoy!

Our new blog post about Apache Ice erg limitations: quesma.com/blog-detail/ap…

At #IcebergSummit 2025, Ryan Blue unveiled Iceberg beyond Java, plus the path to Table Spec V3 & forward to V4. Przemysław Delewski’s new blog covers Fokko Driesprong on Pylceberg, Matt Topol on Go, Julien Le Dem on modular DBs. Essential read for next-gen data platforms. Link👇

Iceberg Table V3 is coming: dremio.com/blog/apache-ic…

Everything is better when Kawaii 🌸🌸🌸: Titanic survival rates with freshly-released Quesma Charts. app.charts.quesma.com/s/20bvqu At @DataCouncilAI conference in Oakland with Jacek Migdał. #dataViz @QuesmaOrg @jakozaur