Ragavan

@radoshi @HiroFinanceAI Congratulations!!! 💰💰💰🚀🚀🚀

I’m thrilled to announce what I've been working on for the past year: @HiroFinanceAI Private beta starting today: hirofinance.com

Meta COOKED! Llama 4 is out! Llama 4 Maverick (402B) and Scout (109B) - natively multimodal, multilingual and scaled to 10 MILLION context! BEATS DeepSeek v3🔥 Llama 4 Maverick: > 17B active parameters, 128 experts, 400B total parameters > Beats GPT-4o & Gemini 2.0 Flash, competitive with DeepSeek v3 at half the active parameters > 1417 ELO on LMArena (chat performance). > Optimized for image understanding, reasoning, and multilingual tasks Llama 4 Scout: > 17B active parameters, 16 experts, 109B total parameters > Best-in-class multimodal model for its size, fits on a single H100 GPU (with Int4 quantization) > 10M token context window > Outperforms Gemma 3, Gemini 2.0 Flash-Lite, Mistral 3.1 on benchmarks Architecture & Innovations > Mixture-of-Experts (MoE): First natively multimodal Llama models with MoE > Llama 4 Maverick: 128 experts, shared expert + routed experts for better efficiency. Native Multimodality & Early Fusion: > Jointly pre-trained on text, images, video (30T+ tokens, 2x Llama 3) > MetaCLIP-based vision encoder, optimized for LLM integration > Supports multi-image inputs (up to 8 tested, 48 pre-trained) Long Context & iRoPE Architecture: > 10M token support (Llama 4 Scout) > Interleaved attention layers (no positional embeddings) > Temperature-scaled attention for better length generalization Training Efficiency: > FP8 precision (390 TFLOPs/GPU on 32K GPUs for Behemoth) > MetaP technique: Auto-tuning hyperparameters (learning rates, initialization) Revamped Pipeline: > Lightweight Supervised Fine-Tuning (SFT) → Online RL → Lightweight DPO > Hard-prompt filtering (50%+ easy data removed) for better reasoning/coding > Continuous Online RL: Adaptive filtering for medium/hard prompts All model on Hugging Face - time to COOK!

BREAKING: Meta's Llama 4 Maverick just hit #2 overall - becoming the 4th org to break 1400+ on Arena!🔥 Highlights: - #1 open model, surpassing DeepSeek - Tied #1 in Hard Prompts, Coding, Math, Creative Writing - Huge leap over Llama 3 405B: 1268 → 1417 - #5 under style control Huge congrats to @AIatMeta — and another big win for open-source! 👏 More analysis below⬇️

@simonw @Ahmad_Al_Dahle Not yet...

@Ahmad_Al_Dahle Does Maverick have the 10 million token context too?

Introducing our first set of Llama 4 models! We’ve been hard at work doing a complete re-design of the Llama series. I’m so excited to share it with the world today and mark another major milestone for the Llama herd as we release the *first* open source models in the Llama 4 collection 🦙. Here are some highlights: 📌 The Llama series have been re-designed to use state of the art mixture-of-experts (MoE) architecture and natively trained with multimodality. We’re dropping Llama 4 Scout & Llama 4 Maverick, and previewing Llama 4 Behemoth. 📌 Llama 4 Scout is highest performing small model with 17B activated parameters with 16 experts. It’s crazy fast, natively multimodal, and very smart. It achieves an industry leading 10M+ token context window and can also run on a single GPU! 📌 Llama 4 Maverick is the best multimodal model in its class, beating GPT-4o and Gemini 2.0 Flash across a broad range of widely reported benchmarks, while achieving comparable results to the new DeepSeek v3 on reasoning and coding – at less than half the active parameters. It offers a best-in-class performance to cost ratio with an experimental chat version scoring ELO of 1417 on LMArena. It can also run on a single host! 📌 Previewing Llama 4 Behemoth, our most powerful model yet and among the world’s smartest LLMs. Llama 4 Behemoth outperforms GPT4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks. Llama 4 Behemoth is still training, and we’re excited to share more details about it even while it’s still in flight. A big thanks to all of our launch partners (full list in blog) for helping us bring Llama 4 to developers everywhere including @huggingface, @togethercompute, @SnowflakeDB, @ollama, @databricks and many others👏 This is just the start, we have more models coming and the team is really cooking – look out for Llama 4 Reasoning 😉 A few weeks ago, we celebrated Llama being downloaded over 1 billion times. Llama 4 demonstrates our long-term commitment to open source AI, the entire open source AI community, and our unwavering belief that open systems will produce the best small, mid-size and soon frontier models. Llama would be nothing without the global open source AI community & we are so ready to begin this next chapter with you. 🦙 Read more about the release here: llama.com, and try it in our products today.

Excited to ship the first set of Llama 4 models today. llama.com/4

Excited to share a research breakthrough from our team. It's fun, it's personal, it's customizable. Huge congrats to the team that worked hard to get to this milestone. ai.meta.com/blog/movie-gen…

@dalmaer So cool to see this, @dalmaer. This sounds like such a great fit. Wishing you the best.

This is my personal journey to Augment, and our vision for how to best help developers. blog.almaer.com/introducing-au…

@mattshumer_ It's been humbling to see the community come together on Llama 3. Thank you. 🙏🙏

It's been a week since LLaMA 3 dropped. In that time, we've: - extended context from 8K -> 128K - trained multiple ridiculously performant fine-tunes - got inference working at 800+ tokens/second If Meta keeps releasing OSS models, closed providers won't be able to compete.

@deviparikh Proud of you, @deviparikh! Going to miss you at Meta, but so so so excited for you :)

Update: I left Meta yesterday. After 7.5 years. I am sad, nervous, and excited. Sad because I'll miss Meta! I've felt tremendously valued my entire time at Meta (first in FAIR and recently in GenAI). I'll miss the people and being in the thick of things. Nervous because who in their right mind walks away from the job I had in times like these (leading research efforts in generative media and multimodal LLMs)?! And excited for new experiences :) Stay tuned for when I have more to share!

(1/6): We are entering the Age of Global Resilience. And today with my partner and @generalcatalyst MD Paul Kwan, we’re publishing our thesis on what this is and why we’re all-in: generalcatalyst.com/perspectives/b…

@kevin @mikeyk Great to see this!

It's been a minute, but @mikeyk and I are back at it with Artifact - a personalized news feed using the latest ai tech. Visit artifact.news to sign up and join the community.

@zck But did you use Dall-E to generate the circle though?

drawing a big circle that says "GPT-4" on it and constantly looking back at the audience for approval like a contestant on the price is right

What are the AI-native frames that will define how consumers interact with intelligent software systems? What are the AI-native nouns & verbs that will form the vocabulary of this next generation of products? Are you building these today? We’d love to chat. Cc @generalcatalyst

“Desktop” "Window" “File” “Program” "Download" "Copy/Paste" "Save" "Website" "Tab" “Homepage” "Link" “Online” "Browse" "Search" "App" "Homescreen" “Feed” “Notifications” "Swipe" "Share" "Message"

If 2022 was the year when many talented founders went down the web3 rabbit hole, 2023 will be the year when talented product people build AI-native products & businesses. Here’s why: