sajal
18.8K posts









We're open-sourcing two frameworks for evaluating Indian ASR, and a full guide on evaluation across 22 languages. WER (Word Error Rate) and CER (Character Error Rate) were built for languages like English. Indian languages behave differently. Formal and colloquial forms coexist. English words move fluidly between scripts. In this guide, we introduce four complementary metrics LLM-WER, LLM-CER, Intent Score, and Entity Preservation Score. In combination they capture a more complete view of Indian ASR system performance. Read more: sarvam.ai/blogs/evaluati… Both frameworks are available to use and build on for evaluating Indian ASR in real-world settings. Open-source frameworks: LLM-WER/CER: github.com/sarvamai/llm_w… Intent + Entity Score: github.com/sarvamai/llm_i…

Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software. It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans. anthropic.com/glasswing










