Distributed Systems

Retries in distributed systems assume transient failure -- the same request will eventually succeed. Retries with LLMs just re-roll the dice. Same input, different latent trajectory.

« Don’t ever make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence much too much credit. » (Linus Torvalds) « In my field of research (machine learning, and especially deep learning & neural nets), [theory lags practice] is a truth I have experienced first-hand. » (Yoshua Bengio) Watt invented the engine long before scientists conceived thermodynamics. We built electric circuits before scientists founded electromagnetism. We hacked computers together and then founded computer science. We created large language models and in the future, we will understand why they work. Daniel Lemire, "Theory lags practice," in Daniel Lemire's blog, January 7, 2015, lemire.me/blog/2015/01/0…

The approach is such a clean way to bridge the gap between empirical observation and formal theory. In distributed systems, we often struggle with “emergent” behaviors we can’t fully prove.. this level of precision in tracking true posterior is huge step for interpretability

New work: Do transformers actually do Bayesian inference? We built “Bayesian wind tunnels” where the true posterior is known exactly. Result: transformers track Bayes with 10⁻³-bit precision.
And we now know why. I: arxiv.org/abs/2512.22471 II: arxiv.org/abs/2512.22473
🧵

@TheValueist The shift toward disaggregated prefill and decode is essentially turning LLM inference into a classic distributed systems problem: managing state transfer overhead vs. compute efficienc! Spot on!

@copyconstruct Datadog's success is proof that developer experience matters more than technical superiority. I am more surprised by MongoDB!

Sometimes, it’s hard to believe the Datadog is a 55billion public company. For reference, Cloudflare’s valuation is 67billion, and Cloudflare’s the backbone of the entire internet. Datadog needs to be a special case study, because no other monitoring company even comes close.

Back after a long break. Turns out distributed systems are still hard. AWS had 3 major outages in 2025 alone. Some things never change.

I wrote a few words to mark the publication of @unmeshjoshi's excellent book after a fruitful four-year journey martinfowler.com/books/patterns…

If databases and their internals fascinate you -- this is quite a treasure trove of lectures from 2014 to now! db.cs.cmu.edu/seminar2023/ Especially the Time-Series one from 2017!

Thrilled by the insightful @dist_sys meetup at @nutanix Pune office, July 2023! 🚀 A shout-out to all participants for making it remarkable. Exciting lineup in store for upcoming meetups. Stay tuned for your next meetup - meetup.com/Distributed-Sy…

@dist_sys We’re setting up a Distributed System Testing group in London and will be having our first event on 21st June. If you would like to meet like minded people, check out the details and sign up! meetup.com/innovations-in…

Thoroughly enjoyed attending this demo of TLA+ by Markus who works on it at MSR. I liked how he incrementally built up the problem and its pitfalls, how TLA+ scores over manual reasoning and tests, implications of state space explosion, invariants, liveness, starvation ... etc

Meetup Recording is now available youtu.be/lsgDQhMC19o @lemmster

@hillelogram Come to our hang out on discord? Would love to run a session with the group

If you're starting out TLA+ or Alloy and want me to give your spec a quick once-over for obvious improvements feel free to reply or DM me, I like helping beginners

@indygupta We would love to host one of the lectures publicly for the community:)

False! Reports of my Distributed Systems class having 250 students are grossly exaggerated. Actual number of enrolled students in my class Distributed Systems Fall 2022 is only 539. (This is an elective, and not a required course.)

And we are on! It was fun coordinating the meet-up with @dist_sys. I am so excited to spend more time on TLA+.

All set for the distributed systems in Bengaluru meet-up with @lemmster on TLA+. Thanks for organizing @dist_sys

We have a location! Thanks to @thoughtworks Bengaluru meetup.com/distributed-sy… @lemmster @vaibhaw_vipul @unmeshjoshi @ShripadAgashe

By age 30 you should have a group of friends that talk you out of distributed systems