pseudotensor

@an_vo12 Acts like a feature not a bug. The anomalous legs are properly discounted as not real enough for that particular animal. Do it for an unknown animal for which leg counting is not a strongly known prior.

🚨 Our latest work shows that SOTA VLMs (o3, o4-mini, Sonnet, Gemini Pro) fail at counting legs due to bias⁉️ See simple cases where VLMs get it wrong, no matter how you prompt them. 🧪 Think your VLM can do better? Try it yourself here: #example-gallery-section" target="_blank" rel="nofollow noopener">vlmsarebiased.github.io/#example-galle… 1/n #ICML2025

For GAIA, the dataset genspark.ai used is called a "validation" data set that is leaked all over internet. They should submit to the official test set. They also excluded results from Trase and H2O.ai referring to a 4 month old result as "previous SOTA": huggingface.co/spaces/gaia-be… and h2o.ai/blog/2025/h2o-…

Meet Genspark Super Agent - a fast & reliable general AI agent! Check it out: genspark.ai

@manusai Need to post to GAIA test set. Your agent may be finding the many validation datasets online that allow the agent to cheat and get high validation score.

Introducing Manus: the first general AI agent. Try Manus today and see the future of human-machine collaboration: manus.im

@ylecun 1 year later, LLMs get 65% on test set vs. degree-holding humans at 92%. Not bad year I think. h2o.ai/blog/2024/h2o-…

GAIA: A benchmark for general AI assistants, by a team from Meta-FAIR, Meta-GenAI, HuggingFace, and AutoGPT. Current Auto-Regressive LLMs don't do very well.

@MFarajtabar E.g. a paper you didn't cite uses gsm hard and shows major drop in performance: arxiv.org/abs/2406.07394

1/ Can Large Language Models (LLMs) truly reason? Or are they just sophisticated pattern matchers? In our latest preprint, we explore this key question through a large-scale study of both open-source like Llama, Phi, Gemma, and Mistral and leading closed models, including the recent OpenAI GPT-4o and o1-series. arxiv.org/pdf/2410.05229 Work done with @i_mirzadeh, @KeivanAlizadeh2, Hooman Shahrokhi, Samy Bengio, @OncelTuzel. #LLM #Reasoning #Mathematics #AGI #Research #Apple

@MFarajtabar Similar results for gsm8k hard huggingface.co/datasets/reaso…, which changes all the numbers and if LLMs were just applying some basic math it shouldn't matter much, but it matters alot.

@MFarajtabar Adding irrelevant items and seeing performance drop isn't new. I remember AI explained channel talking about this year(s) ago and relevant for his Simple Bench.

@shishirpatil_ Why is LMSYS in the name and blog? Seems unrelated to them: lmsys.org

Introducing the Agent Arena by 🦍 Gorilla X LMSYS Chatbot Arena 🎯 How do different agents stack up in tasks like search, finance, RAG, and beyond? Which model is the most effective for agentic tasks? What tools do users prefer? Explore these questions and more! ✏️Blog: gorilla.cs.berkeley.edu/blogs/14_agent… 🏟️Arena: agent-arena.com 📊Leaderboard: agent-arena.com/leaderboard ⚱️2k pair-wise battles dataset: #evaluation-directory" target="_blank" rel="nofollow noopener">github.com/ShishirPatil/g… ❓What model, framework, or tools do users prefer? Which agents excel at financial and numerical analysis? What are the best agents to find the needle in the haystack in massive corpuses of data? Which agents are best integrated with online platforms (Gmail, Yelp, etc)? Agents = LLMs + Tools + Frameworks. With Agent Arena, you can compare combinations of large language models, tools (like code interpreters and APIs), and frameworks (including LangChain, LlamaIndex, CrewAI) to find the best agentic-mix for your needs. With a novel ranking system, we evaluate agents based on their performance in real-time head-to-head tasks, tracking the strengths of individual components, and combined. This provides deeper insights into specific use-cases and allows users to see which agent performs best for their needs. In a world of crowd-sourced evaluations, who evaluates the crowd? With Prompt-Hub, users can publish, upvote, and explore prompts used for agent evaluations, creating a collaborative space for the community. From the Agent Arena team of @NithikYekollu, @arth_bohra, Kai Wen, Sai Kolasani, @infwinston, @ml_angelopoulos, @profjoeyg, Ion Stoica, @shishirpatil_ Come see which agents and models rise to the top! 🚀

🚨 BREAKING: Open-Strawberry aims to recreate OpenAI's o1 as open-source! 🍓 🔓 Democratizing AI 🚀 Accelerating innovation 🌐 Community-driven development Join the revolution: github.com/pseudotensor/o… RT to support open AI! 🔄 #OpenSourceAI #AIRevolution

@MatthewBerman A simple coding agent can do these kinds of things:

The second question blows my mind. It didn't count spaces or non-letter characters. 🍓

@mattshumer_ @GlaiveAI Very first try failed. It's thinking is poor.

I'm excited to announce Reflection 70B, the world’s top open-source model. Trained using Reflection-Tuning, a technique developed to enable LLMs to fix their own mistakes. 405B coming next week - we expect it to be the best model in the world. Built w/ @GlaiveAI. Read on ⬇️:

@clefourrier This was done here among other papers: arxiv.org/abs/2402.01781…

Did you know that, if you evaluate the same model, with the same prompt formatting & the same fixed few-shot examples, only changing ♻️the order in which the few shot examples are added to the prompt ♻️ you get a difference of up to 3 points in evaluation score?

Try mistral.ai mixtral 8*7B at gpt.h2o.ai

Massive thanks to @ykilcher and Open Assistant team for open sourcing their data. We released fully Apache v2 model and projects, some using their amazing data. This includes fully open 20B models. See: github.com/h2oai/h2ogpt .

Largest fully apache2 20B parameter LLM with free chatbot: gpt.h2o.ai Checkout the project at: github.com/h2oai/h2ogpt

20B parameter chatbot speaks on the coming of AGI

huggingface.co/h2oai/h2ogpt-o… #h2oGPT #GPT #OSS #OpenSource ^ 20B parameters Apache 2.0!

@epic4kids @andreakhaid That's not what was asked. They asked hot to prevent child from accessing, like the "learning videos" option but also for "read to me" since it prevents learning to read. I canceled my subscription due to this. On amazon kids, where you can't even turn off videos.

@andreakhaid Hi there! You can turn of the Read-to-Me feature by pressing the green pause button at the lower left-hand corner of the book!

You asked...We listened! Read-to-Me books now offer Follow-Along Word Highlighting to help improve reading skills >> bit.ly/read-to-me-boo…

Sleepy on playground