Suman Dey

The dspy.RLM module is now released 👀 Install DSPy 3.1.2 to try it. Usage is plug-and-play with your existing Signatures. A little example of it helping @lateinteraction and I figure out some scattered backlogs:

2026 is going to be the year of continual learning and RL for LLMs. So we’ve compiled our users’ favorites into a reading list to help you start 2026 strong!

GPT-5.2 is now rolling out to everyone. openai.com/index/introduc…

Introducing Gemini 3 ✨ It’s the best model in the world for multimodal understanding, and our most powerful agentic + vibe coding model yet. Gemini 3 can bring any idea to life, quickly grasping context and intent so you can get what you need with less prompting.  Find Gemini 3 Pro rolling out today in the @Geminiapp and AI Mode in Search. For developers, build with it now in @GoogleAIStudio and Vertex AI.  Excited for you to try it!

🚀 GPT-5.1 is officially here! OpenAI just dropped GPT-5.1 (Nov 12, 2025) and the AI community is buzzing. Here's what you need to know: ✨ Key Features: • Enhanced conversational abilities • Adaptive reasoning • 8 preset personality styles • Instant & Thinking variants • 24-hour prompt caching • Specialized coding models 💡 What users are saying: • "5.1 way better than 5 in coding" - developers loving the improvements • Better instruction following • More adaptable for big tasks • Same API pricing as GPT-5 Sam Altman: "It's smarter, more reliable, and a lot more conversational." 🔥 Already integrated in Cursor & GitHub Copilot.

📦 TOON: The Smart Way to Cut LLM Costs by 30-60% Token-Oriented Object Notation is changing how we build AI apps. Here's what makes it powerful: ✅ 30-60% fewer tokens vs JSON ✅ Perfect for RAG pipelines & AI agents ✅ Works with GPT-5, Claude, Gemini ✅ Built-in validation & schema-aware 🔗 github.com/toon-format/to…

🚀 Announcing the Indic LLM-Arena 🇮🇳 At AI4Bharat (IIT Madras), our mission has always been clear - build open, inclusive, and world-class AI for Indian languages. To further this goal, today, we’re introducing the Indic LLM-Arena, a crowd-sourced, human-in-the-loop leaderboard designed to evaluate Large Language Models (LLMs) for India - by Indians. 🔍 Why we built this? Existing global leaderboards are overwhelmingly English-centric. They don’t test how models perform on Indian languages, code-mixed queries (like Hinglish or Tanglish), or culturally grounded scenarios. We’re closing this gap with a platform built to evaluate models on the three pillars that truly matter for India. 🗣️ Language – Can it truly understand how Indians speak, write, and switch languages? 🌏 Context – Can it respond in a way that fits local culture and real-world needs for both users and companies using these models? ⚖️ Safety – Does it respect India’s social sensitivities and fairness norms? With several sovereign LLM efforts underway under the IndiaAI Mission, this leaderboard aspires to serve as a trusted benchmark for evaluating their capabilities for India. 💡 How it works? Users enter real-world Indian prompts (you can even speak or use our transliteration feature) → Two anonymous LLMs respond → You pick the better one → Thousands of such votes power statistically robust rankings. 🧭 Why this matters? This isn’t just a leaderboard - it’s a public utility for the Indian AI ecosystem: • Developers can benchmark and improve Indic LLMs • Enterprises can choose the right model for their domains • The public helps define what “good” AI means for India 📈 What’s next? We’re expanding to multimodal models (vision + audio), agentic tasks (PDFs, search, tool-use), and domain-wise leaderboards - and as always open-sourcing everything along the way. 💬 Try it. Test it. Challenge it here arena.ai4bharat.org/#/chat We thank Google Cloud for their initial support which enabled this launch! Read our detailed blog here ai4bharat.iitm.ac.in/blog/indic-llm… Let’s define how AI should understand India  together. 📧 Reach us at arena@ai4bharat.org @MiteshKhapra , @partha_p_t , Ashwani Sharma , @harshdhand Neama Dadkhahnikoo , Amrita Kamat , Santosh Kevlani, Santosh Pawar, David Joseph Menezes , @SafiKhan2k , @chinkuhere , @KartikVirendra

OpenAI released IndQA - a new benchmark for evaluating AI systems on Indian culture and languages! 🇮🇳 Key highlights: ✅ 2,278 questions across 12 Indian languages ✅ 10 cultural domains (Architecture, Food, History, Literature, etc.) ✅ Created by 261 domain experts from India ✅ Focuses on culturally nuanced, reasoning-heavy tasks Supported languages: Bengali, Hindi, Tamil, Telugu, Kannada, Malayalam, Marathi, Gujarati, Punjabi, Odia, and Hinglish! This is a major step toward making AI truly work for India's billion+ non-English speakers. Source: openai.com/index/introduc…

Realtime Prompting Guide from OpenAI. #instruction-following" target="_blank" rel="nofollow noopener">cookbook.openai.com/examples/realt…

The AI Engineering book from @chipro is GOAT. But I didn't realize its repo has a goldmine .md of the resources she used to write the book. These are papers and blogs you can use to learn about making LLM apps, prompt engineering, fine-tuning, RAG, and much more.

If you found it useful, reshare it with your network. Follow me → @datawithsuman for more such content on ML, and LLMs! twitter.com/datawithsuman/…

POML by Microsoft offers structure, maintainability, and versatility for prompts. Key features: - Structured Prompting Markup - Comprehensive Data Handling - Decoupled Presentation Styling - Integrated Templating Engine GitHub: github.com/microsoft/poml

10x context for Claude Code, Cursor, and +10 other AI IDEs with open-source memory layer. Explore Cipher → first open-source memory layer for coding agents (currently at 2k+ ⭐ in 1 month). Built by ByteRover team. 🧠 Real-time, context-relevant memory retrieval that adapts to your growing, complex codebase with Semantic Search. 🧠 Dual memory layer that capture what matters for your agent's context: - system 1: programming concepts, past interactions with LLM, business logic. - system 2: reasoning steps of the model). 🤝 Easily share the context across your dev team in real time. 🔌 MCP integration with any IDE you want. Let see Cipher in action 👇 github.com/campfirein/cip…

Prompt Engineering is a critical skill in Applied AI. A top resource to start: promptingguide.ai

We just launched ChatGPT Go in India, a new subscription tier that gives users in India more access to our most popular features: 10x higher message limits, 10x more image generations, 10x more file uploads, and 2x longer memory compared with our free tier. All for Rs. 399. 🇮🇳

Thanks for Reading. Paper Link - arxiv.org/pdf/2507.22887

The "End of User Message" (eum) Position is Consistently Harmful: Across all tested use cases, placing demos afterthe user's query was the worst position. It not only degraded performance but also introduced significant prediction instability, making the model's output volatile and untrustworthy.

How Demos Position in a Prompt impact Performance A fascinating new paper, "Where to show Demos in Your Prompt," reveals a powerful and previously overlooked phenomenon: DEMOS POSITION IN PROMPT (DPP) bias. The core finding is that the placement of demonstration examples in a prompt can alter an LLM's accuracy by up to 20 percentage points and change nearly half of its predictions, even when the content is identical.