Andrew Bean

@MilaNLProc @tiancheng_hu Thanks for having me!

📖 For our weekly @MilaNLProc lab seminar, @tiancheng_hu presented the wrap-up of his research visit with our group. We also have @andrew_m_bean, who presented "LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages."

A real honour and career dream that PRISM has won a @NeurIPSConf best paper award! 🌈 One year ago I was sat in a 13,000+ person audience of NeurIPs '23 having just finished data collection. Safe to say I've gone from feeling #stressed to very #blessed 😁

Super excited to see PRISM recognised as a #NeurIPS2024 best paper. This was an incredible large-scale effort by @hannahrosekirk and fantastic collaborators. If you're interested in human feedback, check it out, there are 100+ pages of detailed insights! 🔥

Big congratulations to my DPhil student @andrew_m_bean on an excellent presentation on benchmarking Olympiad-level linguistic reasoning puzzles in low-resource languages at the Meta Open Innovation Research Community event in London (Oct 29, 2024)! 🎉

🌎Introducing LINGOLY, our new reasoning benchmark that stumps even top LLMs (best models only reach ~35% accuracy)🥴 In a colab between @UniofOxford, @Stanford and UK Linguistic Olympiad puzzle authors, we stress test LLMs on over 90 low-resource and extinct languages...