Anna Geller

#Serverless is still misunderstood. It’s not about servers or lack of them. It’s an operational construct 🧵

Are YOU ready for the Open Source event of the year? #Hacktoberfest starts on the 1st of October, and we have some amazing guests to help you jump on it: @eddiejaoude @kunalstwt @debs_obrien @virtualized6ix @securestep9 ⏰ 29 September 4 pm UTC+1 Join the Space in comments

🚀 @kestra_io 1.0 is live! 🤖 AI Copilot 🧠 AI Agents 🔎 AI-powered docs 🔗 MCP Server 🗺️ New flow dependency graph ✨ No-Code Apps & Tests ✅ Many features out of Beta 🔁 New Git sync ☸️ New Helm charts 🪨 Stable LTS release 👉 Full launch post: go.kestra.io/release-1-0-x

✨ Kestra 0.23 is live! What's new: 🧪 Unit Tests for Flows 🖥️ Multi-panel & No-Code Editor 🔒 Tenant storage isolation 🔍 New UI filters 📊 Custom dashboards 🐍 Python deps caching 🔗 Git sync for dashboards/apps 🧩 New plugins: Salesforce, HubSpot, Ollama, LangChain

I am keen to show you with a few lines of YAML config I am able to monitor my website and create a GitHub Issue if it is down by using the Open Source project @kestra_io Kestra is the Open Source Orchestration Tool with a large ecosystem of plugins, with these plugins you can connect them together like lego and building anything! Try it yourself, GitHub repo link below...

How to orchestrate and automate DevOps workflows. (100% open-source, 18k+ stars, works at scale) 🧵🔖

🚀 Featured on @devopstoolbox! Gone are the days of painful deployments and manual scripts. Automate EVERYTHING from CI/CD to OpenAI using Kestra. Watch the magic happen: youtu.be/OePlSQGtbJs?fe…

Join my session today at Airbyte's move(data): Namespaces Are All You Need 🗓️ 20 March, 2:40–2:50 pm PST (10:40–10:50 pm CET) Link to join: movedata.airbyte.com/event/namespac…

Join our Product Lead, @anna__geller , this Thursday at the free online move(data) conference by @AirbyteHQ  for a talk about well-governed data orchestration at scale. Save the date: 20 March 2:40 pm-2:50 pm PST (10:40 pm-10:50 pm CET) movedata.airbyte.com/event/namespac…

If you could only learn one thing that will be relevant for the next 10-20 years, focus on learning how to deal with data. The future is not about faster hardware, smarter algorithms, or better ideas. The future is about DATA, and those who know how to deal with it will stay relevant much longer than anyone else. I recorded a video to show you how easy it is to get started. In the video, I'm using @kestra_io. For a long time, I was a fan of AirFlow. Then, I moved to AWS Step Functions. Today, I only use Kestra. Kestra is open-source (repo link below) and kind enough to sponsor my work. The video will show you how easy it is to do the following: 1. Run Kestra locally (literally, one command) 2. Build a simple flow 3. Run Python scripts as part of your flow 4. Connect to HuggingFace models If you have never built a data pipeline, open Kestra's Quick Start Guide and follow their examples. (I think it will take you one weekend to feel comfortable with the application and build the courage you need to get into more serious work.)

⚡️ Kestra 0.20 is live! ⚡️ 👉kestra.io/blogs/release-… ⬥ Build custom UIs with Apps ⬥ Data isolation across teams ⬥ In-app announcements ⬥ Flow-level SLAs ⬥ Skip tasks with runIf ⬥ Enhanced flow triggers ⬥ Pause flows mid-run ⬥ Many new plugins ⬥ UI Bookmarks

Hey friends, @kestra_io is live on Product Hunt! If you’ve found it helpful or what we’re building resonates with you, we’d love your support. Your upvote means a lot!💜 producthunt.com/posts/kestra

By the way, we're launching Kestra on Product Hunt this Friday, November 29th! We'd be grateful if you could support us with an upvote—it means a lot! Thanks so much in advance! 🙏 Click "Notify me" here: producthunt.com/products/kestra

"How do you bridge the gap between custom code and a user-friendly UI in data orchestration?" That's what we explored on the Data Engineering Podcast—just in time for Thanksgiving! Thanks to Tobias Macey for having me on the show. Gobble it up! 🦃 youtu.be/uOzfWBKU2YI?si…

Really interesting article on how to build #serverless data pipelines with @kestra_io, Modal, dbt, and BigQuery. It got me digging deeper into Kestra and I’m seriously impressed by their open-source orchestration platform. The workflow UI is super slick and you can easily integrate and build automations with just about any data stack you can think of. They’re also working on a cloud version that looks like it’ll be completely serverless! 🙌 shortclick.link/vktmwr

Data pipelines will put you in the top 1% of the market. If you could only learn one skill for the next decade, I can't think of anything more critical than learning to move and process data at scale. I like to tell people I'm a Machine Learning Engineer, but in reality, 90% of the value I produce comes from my ability to move data around consistently and correctly. In the field, we like to use the term "orchestration" when talking about coordinating workflows that move and process data. At a high level, there are three main steps you need to worry about: 1. Getting the data from its source 2. Processing and cleaning that data 3. Delivering the cleaned data to the right place You might have also heard about "ETL" (Extract, Transform, Load). That's how most people refer to the process above. Of course, building a simple ETL system isn't complex; most developers can do it without too much trouble. The problem is designing resilient, scalable, and fault-tolerant systems. You can't code your way to a production-ready orchestration platform (ask me how I know.) I started with AirFlow and eventually moved to @kestra_io because of its event-driven architecture. Event-driven means you can kick off a workflow automatically based on different triggers. For instance, when somebody uploads a new file to a folder, an app updates a database table, or there's a new message in a queue. It's hard to summarize everything you get from Kestra, but here are some of the highlights: • Kestra is free and open-source • You install it from a Docker container • Workflows as Code using YAML <--- this is awesome • Scales to millions of executions • It integrates with every cloud platform you've seen • Language agnostic (but I still like Python the most) Here is a link to their GitHub repository: shortclick.link/4ls02n Here are the three things I recommend: 1 - Take a look at their live demo in their GitHub repo 2 - Build a simple workflow (it will take 5 minutes) 3 - Talk to your boss. Where can you plug this into your company? I started using Kestra at the height of the pandemic. It's an awesome tool, and I'm proud that they are sponsoring my writing. I hope you find it helpful as well.

companion blog post kestra.io/blogs/serverle…

small video demo: youtu.be/Wqz7CZudqNo?si…

📢 New end-to-end Data Engineering project Excited to share a small reference architecture project for a (mostly) Serverless Data Stack using BigQuery, dbt, @modal_labs, with @kestra_io orchestrating all components: github.com/kestra-io/serv…

New in @kestra_io: conditional inputs to build interactive workflows! ⚡️ We are close to reaching 10k stars on GitHub. If you’d like to help us hit that milestone, give us a ⭐️ or share this post: lnkd.in/exbpXQSR

I've been doing some work with @kestra_io recently, and am so excited to see them raise funding to bring their declarative, IaC vision for workflow orchestration to everyone! They have 7.7K stars on GitHub already, let's get them trending: github.com/kestra-io/kest…