Metaflow

Dare I say that Metaflow is making me like Kubernetes at least a little bit again.

You can still register for Nebius × @OuterboundsHQ webinar on building custom LLMs: nebius.com/events/webinar…. Join us tomorrow at 10 AM PDT. We’ll cover a modern stack for developing domain-specific models using the latest post-training tricks, including custom reward functions.

Ever wondered how much CPU, memory, GPU, disk, or network traffic your @MetaflowOSS steps use and how to optimize resource allocation? The open-source resource-tracker Python package does that (and more) with seamless #metaflow integration! Read more at sparecores.com/article/metafl… 🚀

Metaflow simplifies orchestration, optimizes compute and integrates with Airflow to streamline workflows. @SavinGoyal from @OuterboundsHQ shares insights on scaling infrastructure and lessons from his time at @netflix. Link in the comments. #AI #MachineLearning #Airflow

Nebius and @OuterboundsHQ form strategic partnership to deliver a full-stack solution for business-critical AI systems 🔀 Read the blog to find out more and start using the integration: nebius.com/blog/posts/neb… That’s right: Nebius AI Cloud, the ultimate cloud for AI innovators, and Outerbounds, an AI/ML platform powered by the popular open-source framework @MetaflowOSS, are now integrated as part of a broader technology partnership. This integration brings great value to machine learning engineers and organizations who can now seamlessly deploy and manage their AI and ML models at scale, leveraging advanced infrastructure management tools, robust security frameworks and high-performance computing capabilities.

@rblalock I love @MetaflowOSS docs. Very well organized, clear, easy to navigate.

If you're in the Bay Area, join us tomorrow evening at GitHub's SF office! Our Eddie Mattia will be discussing fine-tuning of LLMs using Metaflow, covering models ranging from 1 billion to 405 billion parameters 💪 RSVP @ hubs.la/Q02-YQCW0

AI meetup on Dec 10 at GitHub SF! Outerbounds' Eddie Mattia will cover cost-efficient fine-tuning of LLMs (1B–405B params) using Metaflow, showing how it integrates with workflows and cloud tools. RSVP: hubs.la/Q02-42Cl0

@anushkmittal big time if it good for Netflix, is good for the rest of the world, for sure.

I reimplemented the entire curriculum of my Machine Learning program. I'm now using @OuterboundsHQ and @MetaflowOSS as the backbone of every pipeline in the program. It's a combination of open-source tools with technology that supports developing scalable, real-world systems. Building real-world ML/AI systems is as challenging today as it's ever been. Here are some fundamental questions: • How do you continuously update systems in production safely? • How do you access data fast and securely? • How to you keep parallel production and staging environments? • How do you manage dependencies and security updates? • How do you react to new data and trigger continuous retraining? • How do you monitor the performance of your system? @OuterboundsHQ and @MetaflowOSS were born at @Netflix. Very few companies in the world have more experience at implementing AI/ML at such a huge scale. It's been a lot of work, but I'm extremely excited about training the next generation of Machine Learning Engineers using the best tools in the world.

Distributed training just got easier on Outerbounds! Now generally available, it supports multi-GPU setups with [@]torchrun and [@]metaflow_ray. Perfect for large data and models. Train efficiently, even at scale! 💡 #AI #DistributedTraining

Join us at #KubeCon for a healthy dose of AI Infrastructure on Kubernetes 🏗️

🚀 Epic #MLMixer at Exploratorium! 🌌 Insightful talks by @vtuulos, @sh_reya 🎓, @GokuMohandas of @anyscalecompute, @nikhilaravi of Meta, & @sarahcat21 of @AmplifyPartners! Fun & deep dives into genAI & LLMs! Thanks @OuterboundsHQ & AGI House for hosting! 🌟 #AI 🤖

What a panel! On the past, present, and future of genAI w/ - @nikhilaravi of meta - @GokuMohandas of @anyscalecompute - @sarahcat21 of @AmplifyPartners - academic cal bear eecs phd @sh_reya 🤌

Kicking off machine learning @ the exploratorium tonight is @vtuulos, @OuterboundsHQ CEO! Talking modeling, deployment, versioning, orchestration, compute, data, oh my!

Unsere #MachineLearning (ML)-Ingenieur:innen und Datenwissenschaftler:innen haben kürzlich ein neues ML-Modell entwickelt. Welche Rolle hierbei #Metaflow spielt und vor welchen Herausforderungen sie standen, erzählen sie in unserem Tech-Blog. ➡️ pro7.de/35a

A major release, Metaflow 2.9, is out today 🎉 New: Build large, advanced ML systems by connecting flows and make them react to events in real-time! 🛰️ outerbounds.com/blog/metaflow-…

Our (boring) prediction: #LLMs and #AI will become tools amongst other tools in the companies' tech stack (albeit super powerful ones 🦸). @MetaflowOSS helps manage the whole ML stack, LLMs included. Read more here: outerbounds.com/blog/llm-infra…

✍️ How to train and deploy a RecSys solution ✅ preprocess the H&M dataset and train a Two-Tower model w/ NVIDIA Merlin ✅ orchestrate training using Metaflow ✅ evaluate with RecList ✅ store predictions in a PaaS key-value store and serve w/ AWS Lambda medium.com/nvidia-merlin/…

Use #Airflow and want to ship better #ML & #DataScience projects faster? All the goodness of #Metaflow, now on top of your nearest Airflow cluster!