Emmanuel Darras

Kestra: Easily Create AI Agents That Can Automate Anything! Opensource n8n Alternative!: youtu.be/ZjcorAgfn3w

Are YOU ready for the Open Source event of the year? #Hacktoberfest starts on the 1st of October, and we have some amazing guests to help you jump on it: @eddiejaoude @kunalstwt @debs_obrien @virtualized6ix @securestep9 ⏰ 29 September 4 pm UTC+1 Join the Space in comments

How to orchestrate and automate DevOps workflows. (100% open-source, 18k+ stars, works at scale) 🧵🔖

The UNDERRATED Open Source Powering My HomeLab // Kestra youtu.be/OePlSQGtbJs?si… via @YouTube

Check out my latest article: Kestra: the unified orchestration platform, a simpler alternative to complex and fragmented tools. linkedin.com/pulse/kestra-u… via @LinkedIn

🎥 Working with orchestration, automation, or data pipelines? Kestra’s YouTube channel has hands-on demos, deep dives, and real-world use cases to help you build better workflows. Check it out & subscribe! 👇 @kestra-io" target="_blank" rel="nofollow noopener">youtube.com/@kestra-io

DeepSeek didn't share its training data with the world. That's the only thing they kept to themselves. Forget about hardware, algorithms, and ideas. Data is all that matters. There's a lesson here: If you could only learn one thing, focus on learning how to deal with data, build orchestration pipelines, and do it all at scale. This is a top 1% skill that will pay for decades. In the field, we use the term "orchestration" when coordinating workflows that move and process data. At a high level, there are three main steps you need to worry about: 1. Getting the data from its source 2. Processing and cleaning that data 3. Delivering the cleaned data to the right place You might also have heard about "ETL" (Extract, Transform, Load), which is how most people refer to the process above. Of course, building a simple ETL system isn't complex; most developers can do it without too much trouble. The problem is designing resilient, scalable, and fault-tolerant systems. You can't code your way to a production-ready orchestration platform (ask me how I know.) I started with AirFlow and eventually moved to @kestra_io because of its event-driven architecture. Kestra is open-source (repo link below), awesome, and kind enough to sponsor my work. Their event-driven architecture means you can kick off a workflow automatically based on different triggers. For instance, when somebody uploads a new file to a folder, an app updates a database table, or there's a new message in a queue. Here are some of the highlights: • Kestra is free and open-source • You install it from a Docker container • Workflows as Code using YAML <--- this is awesome • Scales to millions of executions • It integrates with every cloud platform you've seen • Language agnostic (but I still like Python the most) Here are the three things I recommend: 1 - Take a look at Kestra's live demo in their GitHub repo 2 - Build a simple workflow (it will take 5 minutes) 3 - Find a way to plug this into your company

How do you manage complex data workflows? Meet Kestra - the open-source data orchestration platform that's changing how developers approach workflow automation. Key things you should know: - Open-source (check out the GitHub repo) - Event-driven orchestration platform - Designed for large-scale, complex workflows You can run Kestra locally to test it out. All it takes is a curl command to grab the docker-compose file, and then you can spin up the Kestra instance. What makes it interesting (at least to me) is that all workflows can be stored as code and version controlled. I hate dealing with clunky UIs for building workflows, so this is a massive improvement. I can write a declarative YAML file to describe my entire workflow. Plugins are available for all the popular third-party services - AWS, Azure, GCP, and more. What are some good use cases for Kestra? For example, you can orchestrate complex ETL processes. Define each step in your workflow and schedule its execution. Retries, timeouts, and error handling are built in - so you can make your workflows resilient. Learn more here: fnf.dev/4awnhCx

Workflow automation is hard! Not anymore. Kestra is a game-changer for simplifying workflow automation. Let me show you: From data pipeline workflows and infrastructure workflows to general process automation, managing workflows across multiple tools becomes complex, tiring, and inefficient. With Kestra, you get a unified orchestration platform designed to make automation simple, scalable, and effective. It's an open-source, event-driven orchestration platform that simplifies workflows at any scale. Here’s what makes Kestra stand out: 🔹 Declarative approach Define workflows as code using YAML—easy to write, understand, version, and manage. No more guessing how processes fit together. This makes things so much easier. 🔹 Event-driven workflows Easily trigger workflows based on events, such as data updates or system changes. Kestra’s event-driven architecture ensures your processes execute in real time and in the correct order. 🔹 Intuitive user interface A powerful UI lets you visualize and manage workflows in real time. You can monitor, troubleshoot, and adjust processes much easier. Build workflows directly from the UI. 🔹 Plugin ecosystem With over 600+ built-in plugins, Kestra integrates with your favorite data sources, APIs, and cloud services, making it flexible and extensible. 🔹 Language-agnostic workflows at any scale Run tasks in any language and easily integrate your stack. Kestra orchestrates workflows of any size, from small tasks to enterprise-level pipelines. It’s hard to summarize everything Kestra offers, but here are some of the highlights: • It's free and open-source • Simple install from a Docker container • Workflows as Code using YAML • Scales to millions of executions • Language agnostic • Huge range of cloud platforms & data tool integrations Here are three things I recommend: 1) Take a look at Kestra's live demo in their GitHub repo 2) Build a simple workflow (it will take 5 minutes) 3) See where Kestra can optimize workflows in your company and projects Check it out: drp.li/kestra-z7td

🚀 Watch me create a #DevOps Schedule API Health Check + #Slack Alert workflow in just 5 seconds! ✨ Features: • Daily server health monitoring • Automated Slack notifications • Simple microservices integration 🔗 @kestra_io GitHub: bit.ly/49zJajT

I'm attending Live Demo: Smart Workflow Orchestration. Join me today. linkedin.com/events/livedem…

I'm attending Live Demo: Smart Workflow Orchestration. Join me on December 17. linkedin.com/events/livedem…

Like what we do? Leave us a review and help us spread the word! producthunt.com/products/kestr…

⚡️ Kestra 0.20 is live! ⚡️ 👉kestra.io/blogs/release-… ⬥ Build custom UIs with Apps ⬥ Data isolation across teams ⬥ In-app announcements ⬥ Flow-level SLAs ⬥ Skip tasks with runIf ⬥ Enhanced flow triggers ⬥ Pause flows mid-run ⬥ Many new plugins ⬥ UI Bookmarks

@svpino Thank you @svpino Your post is awesome

Here is the link to their Product Hunt launch: shortclick.link/wg0wu2 From here, you'll find their GitHub repository. Thanks to the Kestra team for sponsoring my writing!

Data pipelines will put you in the top 1% of the market. If you could only learn one skill for the next decade, I can't think of anything more critical than learning to move and process data at scale. I like to tell people I'm a Machine Learning Engineer, but in reality, 90% of the value I produce comes from my ability to move data around consistently and correctly. In the field, we like to use the term "orchestration" when talking about coordinating workflows that move and process data. At a high level, there are three main steps you need to worry about: 1. Getting the data from its source 2. Processing and cleaning that data 3. Delivering the cleaned data to the right place You might also have heard about "ETL" (Extract, Transform, Load), which is how most people refer to the process above. Of course, building a simple ETL system isn't complex; most developers can do it without too much trouble. The problem is designing resilient, scalable, and fault-tolerant systems. You can't code your way to a production-ready orchestration platform (ask me how I know.) I started with AirFlow and eventually moved to @kestra_io because of its event-driven architecture. Event-driven means you can kick off a workflow automatically based on different triggers. For instance, when somebody uploads a new file to a folder, an app updates a database table, or there's a new message in a queue. It's hard to summarize everything you get from Kestra, but here are some of the highlights: • Kestra is free and open-source • You install it from a Docker container • Workflows as Code using YAML <--- this is awesome • Scales to millions of executions • It integrates with every cloud platform you've seen • Language agnostic (but I still like Python the most) Here are the three things I recommend: 1 - Take a look at Kestra's live demo in their GitHub repo 2 - Build a simple workflow (it will take 5 minutes) 3 - Talk to your boss. Where can you plug this into your company? Kestra is launching today on Product Hunt. Check the post below for the link and upvote them!

Hey friends, @kestra_io is live on Product Hunt! If you’ve found it helpful or what we’re building resonates with you, we’d love your support. Your upvote means a lot!💜 producthunt.com/posts/kestra

Kestra: All-in-one open-source orchestration platform that bridges the gap of… producthunt.com/products/kestra via @producthunt