OpenData

How much can a database do with just slatedb and object storage? Turns out: it's a lot. Check out @opendatadbs latest entry: MIT-Licensed Vector Search on SlateDB. Competitive p90 latency and throughput on standard data sets. And all you need is object storage to run it.

[1/5] huge improvements on our timeseries query latency by making it more CPU cache efficient. some interesting takeaways from our new blog drop🧵

It's amazing what you can do if you focus on specific use cases.

Today @responsive_apps is announcing their first DB: OpenData Timeseries. Prometheus-compatible + MIT. 500 bucks handles the same load as a 15-20k managed service. Built on top of SlateDB, too! Who says you can't launch the same day as Opus 4.7?! 🤣 opendata.dev/blog/introduci…

The 2025 blog post series has begun and we're kicking things off with the "official" Kafka Streams upgrade-guide -- check it out for best practices when setting up a new app, rules for safe vs unsafe upgrades, and tips for making the process go smoothly: responsive.dev/blog/topology-…

We are happy to announce that we have completed another SOC2 Type 2 audit along with completing another successful penetration test against our cloud services. You can find the latest reports on our trust center: trust.responsive.dev

(3/3) Application upgrades were by far the #1 requested topic when we asked what the community would like to learn about earlier this year. We hope you find this helpful! responsive.dev/blog/topology-…

This will be the foundation for being able to branch @kafkastreams apps to support seamless blue/green deploys. You can also branch a previous version of the state to debug an issue from the past. Powerful stuff coming to the world of stream processing!

@kafkastreams had an amazing 2024! Check out @BleeGoldman's year in review, where she recaps the 25 major improvements the community made to Kafka Streams in 2024! responsive.dev/blog/kafka-str…

It's remarkable how popular Kafka Streams is. Here's data for people searching for solutions to their Kafka Streams problems and landing on @responsive_apps. That's an up-and-to-the-right chart I love! On that note, what's the best book or blog you've read for optimizing website conversions?

Is stream processing interesting because of the new apps it enables, or because it promises better data processing? I'm in the first camp and am proud of the major contributions Responsive has made to the application space. My reflections on what's next responsive.dev/blog/responsiv…

It's said that Silicon Valley is special because the density of smart people leads to chance encounters that don't happen elsewhere. I can attest to that. Here's how a coffee resulted in @responsive_apps building a DB optimized for stream processing in 8 months. (1/n)

Some problems are impossible to solve without stream processing: for instance, did you know that @getmetronome leverages Kafka and @kafkastreams to deliver real time billing features like spend limits at scale? (1/3)

Small feature drop, row-level TTL for Kafka Streams: "The ttl function can use either the key or value, or both, to compute the ttl for that row and override the default ttl. It's also possible to .. only expire specific records." #row-level-ttl" target="_blank" rel="nofollow noopener">docs.responsive.dev/reference/stor…

Is it end of the road for RocksDB in stream processing? Disaggregated state is the clearly superior architecture, with @responsive_apps investing heavily in SlateDB while Flink 2.0 has forked RocksDB.

Embedding RocksDB in stream processors like Kafka Streams causes a world of operational pain. But that’s not the only reason to drop RocksDB: it was built for local disks and, as Warpstream has shown us, local disks are really expensive in the cloud! 👇🏽 Check out the costs of running embedded RocksDB vs traditional disk-centric databases vs a key-value store built with the disk-less SlateDB. ❗The SlateDB service is upto 10x cheaper to operate than other managed key value stores, and is even cheaper to operate than Kafka Streams with embedded RocksDB! 💡 Imagine getting all the benefits of fully managed and instantly autoscaled remote state for LESS than the cost of embedded RocksDB and it’s numerous operational downsides! If you are curious about what Responsive is doing in this space, drop me a line 🙂

Here’s an interesting fact: around 1/3 of the infrastructure cost of running a stateful Kafka Streams app with RocksDB is writing and reading the changelog topic.

The separation of compute and storage is the hallmark of modern systems, and yet both Kafka Streams and Flink embed RocksDB in their compute nodes. This coupling results in severe operational pain, which I believe has held back the adoption of stream processing in general.

How @responsive_apps async processor redefines the scalability limits of @kafkastreams in 20 seconds

The September edition of #CheckpointChronicle is here! It's packed full of content with links to articles from @responsive_apps, @bbejeck, @vanlightly, @UberEng, Expedia, @github, @AirbnbEng, and more! 👉 dcbl.link/cc-sep241