Can Xu

🚀 ArtifactsBench v1.1 is here!🔥🔥🔥 Automated visual/frontend code evaluation benchmark with full transparency! 🎯 94.4% consistency with WebDev Arena 🆕 Major v1.1 updates: Added more models @Alibaba_Qwen @Kimi_Moonshot etc 🔓 100% open-source with complete reproducibility

🚀🚀🚀

🚀Thrilled to introduce #ArtifactsBench! We're bridging the visual-interactive gap in code generation evaluation. Our benchmark uses a novel automated, multimodal pipeline to assess LLMs on 1,825 diverse tasks. An MLLM-as-Judge evaluates visual artifacts, achieving 94.4% ranking consistency with human experts! Moving beyond algorithmic correctness to a true "what you see is what you get" standard. 🌐 Project: artifactsbenchmark.github.io 📄 Paper: arxiv.org/abs/2507.04952 💻 Code: github.com/Tencent-Hunyua… 🗂️ Dataset: huggingface.co/datasets/tence…

Introducing ArtifactsBench 🎨 An MLLM-as-Judge that evaluates AI-generated UI by looking at a live render. Our pipeline captures interactions via temporal screenshots & scores visual/interactive fidelity against a per-task checklist. This achieves a stunning 94.4% ranking consistency with human votes on WebDev Arena! 🤯 We're open-sourcing everything to accelerate user-centric AI. Explore the project: 🌐 Website: artifactsbenchmark.github.io 📄 Paper: arxiv.org/abs/2507.04952 💻 Code: github.com/Tencent-Hunyua… 📊Dataset:huggingface.co/datasets/tence…

Beyond Hunyuan-TurboS's accomplishments, please also pay more attention to the technological innovations. The impact of these innovative endeavors extends much further than the model itself. All these details are recorded in our technical report (arxiv.org/pdf/2505.15431)

📢 Introducing Adaptive Deep Reasoning—a novel approach that dynamically selects between long-chain and short-chain reasoning based on problem complexity, without compromising performance. 🧠 Two-stage training framework: 1️⃣ Mixed Supervised Fine-tuning – Equips the model with both reasoning modes. 2️⃣ Reinforcement Learning – Integrates GRPO with a long-short adaptive group-wise reward strategy that dynamically assesses prompt complexity to provide customized rewards while optimizing reasoning length when appropriate. Uses a logit-based switching loss to optimize the model’s initial token selection, ensuring the right reasoning mode is chosen. ✅ Key benefits: Seamless reasoning mode switching. Maintains long-chain reasoning accuracy while improving efficiency. 📝Technical Details: arxiv.org/pdf/2505.20101

This paper introduces Hunyuan-TurboS, a hybrid architecture that combines different model strengths with adaptive reasoning to optimize responses based on complexity, balancing performance and efficiency. Methods 🔧: → The architecture is a hybrid Transformer-Mamba Mixture of Experts with 128 layers and AMF, MF block patterns. → It uses an Adaptive Long-short Chain-of-Thought Fusion method involving a trained teacher model and reinforcement learning for dynamic strategy selection. → Multi-round Deliberation Learning employs LLM judge ensembles and human experts in a data flywheel for iterative refinement. → A Two-stage Reinforcement Learning process uses Generative Reward Preference Optimization, first for STEM reasoning, then for general instruction following. 📌 Hybrid design effectively balances Transformer reasoning and Mamba efficiency. 📌 Adaptive thinking significantly lowers inference cost by reducing generation tokens. 📌 Multi-stage alignment tailors model capabilities across diverse complex domains right now. ---------------------------- Paper - arxiv. org/abs/2505.15431 Paper Title: "Hunyuan-TurboS: Advancing LLMs through Mamba-Transformer Synergy and Adaptive Chain-of-Thought"

Tencent presents Hunyuan-TurboS - Hybrid Transformer-Mamba MoE (56B active params) trained on 16T tokens - Dynamically switching between rapid responses and deep ”thinking” modes - Overall top 7 on LMSYS Chatbot Arena

✨ Hunyuan-TurboS: Technical Highlights✨ 🥇 Top 8 on LMSYS Chatbot Arena, beat o4-mini and gemini-2.0-flash ⚡ 560B Hybrid-Transformer-Mamba MoE, 180% Speedup 🧠 Adaptive CoT: 50% of top-tier thinking model's output length 📖 Paper: github.com/Tencent-Hunyua… 💻 Demo: huggingface.co/spaces/tencent…

@brawll66 you can try hunyuan-turbos-20250416 on lmsys website lmarena.ai

@CanXu20 is there a way to use Hunyun-Turbos LLM outside of China?

🚀 Introducing Hunyuan-T1 – The Next Leap in AI Reasoning! 🔥 ✅ 4x Performance Boost – Advanced reasoning, faster decoding 🚀 ✅ TurboS Advantage – Long-text understanding & efficient computation ⚡ ✅ Reinforcement Learning-Driven – 96.7% compute dedicated to reasoning 📈 ✅ Benchmark Leading – Comparable or better than R1 across key tests 🏆

HunYuan-T1 is built on the TurboS foundation, which debuted in the LMSYS Chatbot Arena @lmarena_ai and ranked among the global top 15! 🚀

Top 15！ Congratulations to all the collaborators who participated in this interesting competition

Our Mamba MoE model!

@SebastienBubeck @OpenAI @sama Congratulations!

Just started at @OpenAI and I couldn't be more excited to join at this pivotal moment of safe AGI development! Met so many old friends already, talent density of this place is just insane!! Thank you all for the warm welcome, and in particular @sama. Now let the unicorns fly!

🎉Excited to see the core techniques of WizardLM2 has been accepted to #EMNLP2024 and #NeurIPS2024! 👉Automatic Instruction Evolving for Large Language Models (arxiv.org/abs/2406.00770) 👉Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena (arxiv.org/abs/2407.10627) Congrats to all outstanding co-authors. We will not stop our journey towards AGI.

Today, we're releasing LAB-Bench, a set of >2000 evaluations for language models and agents on scientific research tasks in biology. Public models underperform PhD/postdoc-level humans on nearly all tasks. Claude 3.5 Sonnet is the clear frontrunner atm, but long way to go. 1/

Introducing Claude Engineer 2.0, with agents! 🚀 Biggest update yet with the addition of a code editor and code execution agents, and dynamic editing. When editing files (especially large ones), Engineer will direct a coding agent, and the agent will provide changes in batches. Batches are smartly selected based on file complexity. The code execution agent will run the code and check for issues. It can even start processes (like live servers) and end them. It's insanely powerful! 🔥

Mistral releases their first Mamba Model! 🐍 Codestral Mamba 7B is a Code LLM based on the Mamba2 architecture. Released under Apache 2.0 and achieves 75% on HumanEval for Python Coding. 👀 Blog: mistral.ai/news/codestral… Model: huggingface.co/mistralai/mamb… They also released a Math fine-tuning base on Mistral 7B that achieves 56.6% on MATH and 63.47% on MMLU. Blog: mistral.ai/news/mathstral/ Model: huggingface.co/mistralai/math…