Chris Sachs

#realtime applications are easier to build, compose, scale, and reason about when viewed through the lens of cascading state changes, rather than explicit message passing.

The application layer will be the next domino to fall in the transformation from #batch to #streaming. @NstreamIO is leading the charge, standing on the shoulders of @Confluent, #kafka, and #flink—but solving a rather different problem. linkedin.com/pulse/state-st…

I may have picked the worst day in 15 years to tout how bullish I am about the future. But bullish I am. And here's why: linkedin.com/pulse/why-so-m… @NstreamIO @Swim #SwimOS #StreamingData #RealTime #Kafka

Streaming applications are the ultimate destination for #streamingdata. Transitioning from stateless to #stateful—from data-parallel to entity-parallel—makes it possible to run full-fledged applications entirely in-stream.

TiVo is great! But TiVo also loses everything except what you choose to record. Why do we accept such high rates of data loss by always forcing #streamingdata to disk? Vastly more data can be processed in #realtime than can ever be persisted. #SwimOS

There's a whole new class of enterprise technologies debuting right now: from specialized #analytics databases, to #data-oriented middleware, to application development platforms. insideanalysis.com/yourls/1020202… #SWIM #webinar #Analytics @TreeHiveStrat @c9r @swim

Software should be capable of working the way people do. That means in real-time, as streams of conscious state changes, with agency and autonomy of action. @swim #swimos #kafka #streamingdata #streamingapps #realtime #ux

Your data is in motion. Can your application platform keep up? @Swim #StreamingApplications flow #StreamingData through the full application stack—from #Kafka to #RealTime #UX in 1/2 RTT—executing #stateful business logic along the way. Learn more at @Confluent #Current22.

Real-time data processing - the definition of "real" just keeps on changing! insideanalysis.com/yourls/0901202… #SoftWareInMotion #SWIM #dataprocessing #analytics @c9r @swim

Check out this episode of #SWIM to find out about newer technologies like #Kafka and #Flink, older technologies like MQ Series and related messaging systems insideanalysis.com/yourls/0825202… #SoftWareInMotion #SWIM #webinar #datastream #analytics @c9r @swim

I dig it. The trick is to think from first principles, while measuring everything by real-world outcomes, so that you don't reinvent the wheel like this guy!

Running business logic at the nexus of #dataorchestration enables truly real-time intelligence, automation and observability at scale. The trick is to use a streaming application platform like @swim. Can't wait to discuss with @eric_kavanagh on #DMRadio. lnkd.in/eWdAGwrZ

@eric_kavanagh interviews @c9r of @swim, whose company is reinventing the entire #data #supplychain, opening the door to actual real-time in ways never before seen in this industry insideanalysis.com/yourls/07212022 #DMRadio #data #dataorchestration #webinar

@johnson_brad @featurepeek @Netlify Congrats @johnson_brad, and to everyone at @featurepeek. FTW!

@simoncrosby @swim Demultiplexing and splicing together multiple firehoses is an interesting challenge. Especially w.r.t. backpressure propagation, since the partitions and granularity don't align. Would be easier if individual entity streams weren't arbitrarily jumbled together in the first place.

See the problem? (thx @c9r) cc/@swim

@simoncrosby @MatthewEGunter @swim @ApacheFlink @swimlane Sometimes faster than a ping, if it's a propagated message between agents, and we can colocate them in the same partition ;) The fastest messages are the ones you don't have to send over the network. Data locality is a beautiful thing!

@c9r @MatthewEGunter @swim @ApacheFlink @swimlane 1/2 ping surely? Just half RTT courtesy of WARP

OK, trying again #friday... Here's a high level comparison of @swim & @ApacheFlink Grateful for feedback/corrections. Thanks @MatthewEGunter blog.swim.ai/comparing-swim…

@MatthewEGunter @simoncrosby @swim @ApacheFlink @swimlane Can perform incremental map-reduce operations on the web agent link graph in real-time, as states propagate. Memoizing partial reduce products enables continuous, log-time reductions of very large, rapidly changing datasets. No extra shuffling; only recompute what changed.

@simoncrosby @swim @ApacheFlink Seems like JoinMapLanes might be similar: "Every join map lane can be linked to multiple map lanes." @SwimLane("join") JoinMapLane<FormType, KeyType, ValueType> stateJoinMap = this.joinMapLane();

@MatthewEGunter @simoncrosby @swim @ApacheFlink @swimlane An absolutely crucial component of this approach is to demultiplex firehoses, and route each message to specific, granular web agents. We operate on real-time entity state, updated by numerous data sources. We don't directly operate on messages—too little context.