Distributed Systems

Took some notes and tried really hard to capture in brief @cliff_click . Mistakes if any are all mine.

The idea is to optimize the compiler IR with many small, local graph-rewrite rules (“peepholes”) instead of relying only on big global passes. You inspect a tiny region of the Sea-of-Nodes graph, replace it with a simpler equivalent form, and then reprocess neighboring nodes

Listening to @cliff_click present an amazing talk on Sea of Nodes

At Systems Meetup in SF! luma.com/qslm6lhf?tk=OT… Great talk on tile / cuTile and GPU programming in RUST!

what the Fuck arxiv.org/html/2603.2185…

Great read! arxiv.org/html/2604.06566 TLDR; auto-research (@karpathy) applied to databases! Detail - LLMs to discover better database algorithms (buffer management, index selection, query rewriting) by co-evolving not just the solutions, but the evaluation pipelines themselves.

Retries in distributed systems assume transient failure -- the same request will eventually succeed. Retries with LLMs just re-roll the dice. Same input, different latent trajectory.

« Don’t ever make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence much too much credit. » (Linus Torvalds) « In my field of research (machine learning, and especially deep learning & neural nets), [theory lags practice] is a truth I have experienced first-hand. » (Yoshua Bengio) Watt invented the engine long before scientists conceived thermodynamics. We built electric circuits before scientists founded electromagnetism. We hacked computers together and then founded computer science. We created large language models and in the future, we will understand why they work. Daniel Lemire, "Theory lags practice," in Daniel Lemire's blog, January 7, 2015, lemire.me/blog/2015/01/0…

The approach is such a clean way to bridge the gap between empirical observation and formal theory. In distributed systems, we often struggle with “emergent” behaviors we can’t fully prove.. this level of precision in tracking true posterior is huge step for interpretability

New work: Do transformers actually do Bayesian inference? We built “Bayesian wind tunnels” where the true posterior is known exactly. Result: transformers track Bayes with 10⁻³-bit precision.
And we now know why. I: arxiv.org/abs/2512.22471 II: arxiv.org/abs/2512.22473
🧵

@TheValueist The shift toward disaggregated prefill and decode is essentially turning LLM inference into a classic distributed systems problem: managing state transfer overhead vs. compute efficienc! Spot on!

@copyconstruct Datadog's success is proof that developer experience matters more than technical superiority. I am more surprised by MongoDB!

Sometimes, it’s hard to believe the Datadog is a 55billion public company. For reference, Cloudflare’s valuation is 67billion, and Cloudflare’s the backbone of the entire internet. Datadog needs to be a special case study, because no other monitoring company even comes close.

Back after a long break. Turns out distributed systems are still hard. AWS had 3 major outages in 2025 alone. Some things never change.

I wrote a few words to mark the publication of @unmeshjoshi's excellent book after a fruitful four-year journey martinfowler.com/books/patterns…

If databases and their internals fascinate you -- this is quite a treasure trove of lectures from 2014 to now! db.cs.cmu.edu/seminar2023/ Especially the Time-Series one from 2017!

Thrilled by the insightful @dist_sys meetup at @nutanix Pune office, July 2023! 🚀 A shout-out to all participants for making it remarkable. Exciting lineup in store for upcoming meetups. Stay tuned for your next meetup - meetup.com/Distributed-Sy…

@dist_sys We’re setting up a Distributed System Testing group in London and will be having our first event on 21st June. If you would like to meet like minded people, check out the details and sign up! meetup.com/innovations-in…

Thoroughly enjoyed attending this demo of TLA+ by Markus who works on it at MSR. I liked how he incrementally built up the problem and its pitfalls, how TLA+ scores over manual reasoning and tests, implications of state space explosion, invariants, liveness, starvation ... etc