Arroyo

I'm very excited to announce that after 8 months of work, Arroyo 0.15.0 is now available! This is a huge release that brings all of the improvements, fixes, and learnings from running Arroyo at scale to power @CloudflareDev Pipelines, our fully-managed ingestion service.

The news is finally out! Cloudflare has a Data Platform! We're starting with serverless streaming pipelines (powered by @ArroyoSystems), a managed Iceberg Catalog, and a new distributed SQL engine built on @ApacheDataFusio.

Arroyo 0.14.1 is now available! This is a patch release with a number of fixes over 0.14.0, and is a drop-in upgrade: github.com/ArroyoSystems/…

The third SF DataFusion meetup is today at 6! We have a bunch of great speakers including project leader @andrewlamb1111, @camuelg, @LakeSailHQ , and @Greptime. I'll be giving a talk about how Arroyo does streaming incremental view maintenance to support unbounded SQL queries.

@sdil @CloudflareDev Arroyo is remaining open source and will continue to be developed: arroyo.dev/blog/arroyo-is…

@CloudflareDev @ArroyoSystems what's the future for the open source project Arroyo?

Two big things: 1. Pipelines, our brand new streaming ingestion service, is now in public beta. 2. We've acquired @ArroyoSystems so we can bring SQL-based stateful transforms to Pipelines! Ingest tens-of-thousands of records per second right now: blog.cloudflare.com/cloudflare-acq…

@Cloudflare We couldn't be more excited to be building the future of real-time process with Cloudflare! See all of the details on the blog: arroyo.dev/blog/arroyo-is…

We are very excited to announce that we are joining @Cloudflare to bring stream processing to everybody! Arroyo is coming to the Cloudflare Developer Platform as a serverless stream processing system, and will also remain open-source and self-hostable.

Arroyo 0.14.0 is now available! New features in this release include: 🔎 Lookup joins 🪆 Nested updating aggregates {} Struct types 📖 Streaming SQL syntax 🔀 Sink shuffles See the full release notes at arroyo.dev/blog/arroyo-0-…

Arroyo just crossed 4,000 GitHub stars! It's amazing to see the excitement for a new future of open-source stream processing. Thanks to all of our users and contributors for your help getting here!

@v1gnesh @mwylde Here you go! arroyo.dev/feeds/blog.rss

You'd think that the key to being a fast streaming engine is like clever join algorithms, but it's mostly just being really good at JSON. @ArroyoSystems uses Arrow and the arrow-rs JSON decoder. I think it's pretty cool, so I wrote up how it works: arroyo.dev/blog/fast-arro…

Building a near-real-time analytics stack used to take a team months or years. Thanks to modern tech like S3, @redpandadata, @duckdb, and Arroyo, any company can build a sub-minute data lake in under an hour: arroyo.dev/blog/building-…

Arroyo 0.13.1 is now available: github.com/ArroyoSystems/… This patch release includes several fixes over 0.13.0 and is a drop-in upgrade.

Arroyo 0.13.0 is now available, with 📘 Source metadata support 🐰 RabbitMQ streams connector ⚛ Atomic updating outputs 🔐 IAM auth for Kafka ⛓️‍Operator chaining See all of the details in the release post: arroyo.dev/blog/arroyo-0-…

Arroyo 0.12.1 is now available: github.com/ArroyoSystems/… This patch release contains several bug fixes and is a drop-in upgrade for clusters running 0.12.0.

If you missed #p99conf last week, talks are now available to stream on YouTube. I spoke about the design decisions that went into Arroyo's incredible performance: youtube.com/watch?v=7H4CIK… Come for the Rust hot takes, stay for my terrible hand-drawn architecture diagrams 😅

Arroyo 0.12.0 is now available, with support for 🐍 Python UDFS 🐍, plus Protobuf, faster JSON functions, custom state TTLs, and more. See all the details in the release notes: arroyo.dev/blog/arroyo-0-…

Arroyo is coming to #Current24! Come chat with the team and learn about the future of stream processing.

Looking forward to talking about how we dynamically-load DataFusion UDFs in @ArroyoSystems! Because Rust lacks a stable ABI, this is harder than it sounds!