Label Studio

Scaling AI data quality requires more than adding annotators. This guide covers practical workflows for: • onboarding + calibration • overlap and agreement • reviewer feedback loops • disagreement handling • continuous quality evaluation Read the guide: humansignal.com/blog/scaling-a…

@RightShip_ replaced spreadsheet-based review workflows for maritime inspection PDFs with Label Studio Enterprise. Results: • 3× faster annotation • Better SME workflows • More consistent labeling • Repeatable dataset refreshes A strong example of operationalizing industrial AI workflows around complex document data. humansignal.com/customers/righ…

Less admin. More signal. Recent product updates to Label Studio Enterprise tighten the loop between large-scale annotation, review, and quality measurement: → Assign members to projects in bulk → Label long PDFs page-by-page with thumbnail navigation → See label distribution in Analytics — catch class imbalance early → Measure agreement with Consensus or Pairwise methods, built for GenAI evaluation Read the full changelog → humansignal.com/changelog/

Clinical data breaks when relationships break. A single note can contain multiple findings + attributes. If those links don’t hold, the dataset falls apart downstream. How OM1 scaled annotation with Label Studio: • 25K+ tasks, 4.5K patients • +20–30% annotator efficiency • 50% QA cycle time • 50–75% faster issue resolution Structured workflows > downstream fixes. 👉 Read more: humansignal.com/customers/how-…

Generative models don’t have stable “right answers,” so evaluation can’t rely on a single label. Disagreement is part of the signal. This breaks down how consensus helps make that measurable: humansignal.com/blog/consensus…

AI programs often have plenty of dashboards, metrics, and experiments. Even then, it can be difficult to answer what’s working, what needs to change, and what is ready to ship. This blog outlines a simpler way to measure what actually matters so teams can move faster and run systems with more clarity in production. humansignal.com/blog/the-5-met…

LangSmith traces + human judgment = better agentic AI Now you can pull LangSmith traces directly into Label Studio and close the loop on AI quality. Get the template ⤵️ docs.humansignal.com/tutorials/how_…

Our new template for @langfuse gives you a powerful interface for evaluating agentic AI. 🪢 - Filter by user, assistant or tool - Label for issues, verdicts, severity and expected behavior New step-by-step tutorial in the Label Studio docs: docs.humansignal.com/tutorials/how_…

@braintrust AI observability meets human-in-the-loop evaluation with Label Studio! → Bring traces from Braintrust into Label Studio → Label agent output for issues, verdicts, severity and expected behavior → Improve the quality of your agentic AI docs.humansignal.com/tutorials/how_…

@braintrust AI observability meets human-in-the-loop evaluation with Label Studio! → Bring traces from Braintrust into Label Studio → Label agent output for issues, verdicts, severity and expected behavior → Improve the quality of your agentic AI docs.humansignal.com/tutorials/how_…

@braintrust AI observability meets human-in-the-loop evaluation with Label Studio! → Bring traces from Braintrust into Label Studio → Label agent output for issues, verdicts, severity and expected behavior → Improve the quality of your agentic AI Read more: docs.humansignal.com/tutorials/how_…

@braintrust AI observability meets human-in-the-loop evaluation with Label Studio! → Bring traces from Braintrust into Label Studio → Label agent output for issues, verdicts, severity and expected behavior → Improve the quality of your agentic AI docs.humansignal.com/tutorials/how_…

We spent last week at @PyTorch Conference Europe 2026 talking to teams building AI systems already in production. Big shift in the conversations: → Less focus on models → More focus on evaluation, reliability, and real-world performance We also co-hosted an Open Source AI Soirée with Docling, a room full of people sharing practical lessons from the field (see pictures below!) The open source AI community is just getting started 🚀

Kicking off #PyTorchCon Europe 2026 tomorrow in Paris! We’re hosting the Open Source AI Soirée with Docling tomorrow evening. If you’re working on training, evaluation, or production AI systems, it would be great to connect! Register here: luma.com/ya2wihmc

🇫🇷 Heading to the #PyTorch Conference in Paris? We are. Label Studio is sponsoring PyTorch Conference Europe on April 7–8. Come find us if you want to talk: • LLM evaluation • human feedback workflows • dataset quality

🚀 Getting back into sharing more about what we’re building at Label Studio. From LLM evaluation → human-in-the-loop workflows → dataset curation. For more about AI systems, follow along 👇 labelstud.io

We’re excited to sponsor @PyTorch Conf EU! Join the Label Studio and #Docling teams for drinks & bites to talk all things Open Source AI: Tuesday, April 7th at 18:30 CEST Register to attend. Venue is walking distance from Station F on the Seine. luma.com/ya2wihmc

@huggingface + Label Studio SDK = 🔥 • Easily import HF datasets into Label Studio for annotation • Connect HF models with Label Studio for evals, pre-annotations and active learning • Export labeled data for model training & fine-tuning Tutorial ↓ labelstud.io/tutorials/how_…

Our latest open source release: Label Studio 1.23 🔥 • Vector annotation • Interactive task source • Improved data manager Built for real-world data labeling and AI evaluation workflows. Release notes & quick start: labelstud.io/blog/label-stu…

3/ 🔧 Ready to fine-tune your NER model? Follow Mikaela Kaplan’s steps in our latest article to continuously improve your results. Check it out here: labelstud.io/blog/fine-tuni…

2/ 💥 Walk through creating a Label Studio project, connecting GLiNER, and evaluating and creating new labels to improve your model's accuracy.

1/ 🚨 New Tutorial Alert! Dive into the world of Named Entity Recognition with our latest guide on fine-tuning generalist models like GLiNER for better performance.