Gaurav Srivastava

LLMs can do complex math—but struggle with sorting numbers? We announce LLMThinkBench to evaluate how LLMs reason (or overthink) with basic math reasoning. Huge thanks to Dr. Xuan Wang (@xwang174), Dr. Tu Vu (@tuvllms), Dr. Christine Julien (@drcjulien), Aafiya, and Sriram!

Welcome to join our 𝗔𝗖𝗟 𝟮𝟬𝟮𝟱 𝗕𝗶𝗿𝗱𝘀-𝗼𝗳-𝗮-𝗙𝗲𝗮𝘁𝗵𝗲𝗿 (𝗕𝗼𝗙) 𝘀𝗲𝘀𝘀𝗶𝗼𝗻: 𝗕𝗿𝗶𝗱𝗴𝗶𝗻𝗴 𝗛𝘂𝗺𝗮𝗻 𝗦𝘁𝘂𝗱𝘆 𝗮𝗻𝗱 𝗟𝗟𝗠 𝗔𝗴𝗲𝗻𝘁𝘀 𝗳𝗼𝗿 𝗦𝗼𝗰𝗶𝗮𝗹 𝗦𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻! 🌐 Website: lnkd.in/eq5n3vTz @aclmeeting @priyapitre @ctrlgaurav

@priyapitre @ctrlgaurav

🧠 Fully open-source and customizable 👇 Try it: 📦pip install llmthinkbench (pypi.org/project/llmthi…) 🔗 github.com/ctrl-gaurav/LL… 🆕 v0.1.5 is live with better error handling, fallback support, and multi-GPU parallelization! Stay tuned—benchmark results on 50+ LLMs coming soon!

LLMs can do complex math—but struggle with sorting numbers? We announce LLMThinkBench to evaluate how LLMs reason (or overthink) with basic math reasoning. Huge thanks to Dr. Xuan Wang (@xwang174), Dr. Tu Vu (@tuvllms), Dr. Christine Julien (@drcjulien), Aafiya, and Sriram!