jasmine collins
304 posts
@jazco
research scientist @databricks! previously @berkeley_ai phd, @googleai and @metaai. interested in human brains and computer brains 💫
Most AI benchmarks test reasoning in isolation. Real enterprise tasks require grounded reasoning: 1️⃣ Find the right documents 2️⃣ Extract the right values 3️⃣ Perform analyses OfficeQA Pro evaluates this end-to-end. Frontier agents still score <50%. 🧵Paper & details below!
Today we’re introducing OfficeQA, a new benchmark grounded in ~89,000 pages of U.S. Treasury Bulletins that reflects the complex, document-heavy tasks enterprises actually face. Unlike existing benchmarks, OfficeQA measures economically valuable, real-world reasoning: parsing dense tables, navigating scanned PDFs, and retrieving facts across decades of documents. Even strong agents reach only ~45% accuracy, showing how far the field has to go. The benchmark is now open to the community, and the Databricks Grounded Reasoning Cup in Spring 2026 will challenge teams to push these capabilities forward. databricks.com/blog/introduci…
Quite excited that CommonCanvas is JUST out! 🖼️ • First open source text-to-image models trained fully on openly licensed images (SD2 and SDXL architectures) • The dataset, with ~70M openly licensed creative commons images + synthetic captions, is also out (largest dataset on @huggingface 🤯) Huge kudos to @SkyLi0n, @DbrxMosaicAI 🔥 All on @huggingface 🤗 (ofc with a Gradio demo) huggingface.co/common-canvas
@astonzhangAZ where vision
