Todd Lipcon

@metcalfc @rcrowley Ha flashback to 2009

@rcrowley I recent dug up an old tool from Cloudera days. Similar and has the ability to handle refs in the manifest. @tlipcon wrote the original. I’m not sure about the dates, pretty sure it was just a couple years ago….. github.com/metcalfc/crepo

My poor-man's monorepo: rcrowley.org/2025/monorepo/

@andy_pavlo just watched your talk on writing a DB in ebpf. Was surprised you didn't contrast against the "unikernel" approach? Eg run the DB in the kernel relying on virtualization to sandbox it

Unleash the power of your data with our new Spanner features! Introducing Spanner Graph 🤝 Full-text search 🤝 Vector search—the ultimate trio to supercharge your app development → goo.gle/3LMqcLZ

📢 Introducing *NEW* database capabilities across the Google Cloud portfolio! From Spanner Graph to Bigtable SQL, find out how these innovations help you build the next generation of AI-powered apps → goo.gle/3LM6fVB

@AndresFreundTec @TheASF I think there are still some critical infrastructure software projects operating without code review. And of course even code review is no guarantee since you can just have two fake personas reviewing each other's code....

@AndresFreundTec +1 I recall an intense argument with seasoned folks at @TheASF where I was pushing that committing code without code review should be considered unacceptable and getting lots of push back that this hindered community development or some such (imo) rubbish

I am a bit concerned by all the focus on small-ish projects with overwhelmed maintainers. There indeed are a lot of problems in that area.

@MarkCallaghanDB @mituzas @rakyll Also to be clear, the "one big deployment" is highly sharded/regionalized/etc internally so that failures are really well isolated, upgrades happen bit-by-bit, etc, so that the blast radius of any potential issues doesn't look like "one big deployment". Gotta get those 9s!

@MarkCallaghanDB @mituzas @rakyll as for "cluster" scalability, there is basically one big deployment for 95% of Google. From an architectural perspective you could run a transaction that spans any of that data, though of course security and privacy policies disallow such things.

Since I left Google, I have never met a Xoogler that doesn't miss Spanner. Spanner is clearly the real secret sauce.

@tomjtobin @criccomini Y'all still doing impressive stuff with Kudu at Twilio? Haven't kept up with the community much since switching jobs

@criccomini We did this to Kudu - which provides a great storage engine, but no query planner or query language. This project: github.com/twilio/calcite… was designed to create the query plan and tell kudu to execute it.

I had a random DB thought this morning: Are there any DBs that have an interface to send query plans over the wire rather than SQL? What I'm thinking is essentially a protobuf of something like a substrait plan.

Anyone else having flashbacks to early X11 experiences with this new twitter logo? looks an awful lot like `xlogo`

@MarkCallaghanDB @muratdemirbas Yea, that one. Always forget the name:) maybe I can convince folks to do yet another round

[new blog post] F1: A Distributed SQL Database That Scales muratbuffalo.blogspot.com/2023/07/f1-dis…

@muratdemirbas research.google.com/pubs/archive/4…

@muratdemirbas The "spanner 7 years later" paper is a good one too that talks about some further evolution of the systems

@davidtgoldblatt Feel like half my job is repeatedly pushing on 4-dimensional see-saws in hopes that the seesaw eventually sinks a few millimeters into the playground dirt

@tsunanet What's the cheat code for 30pb quota?

@eatonphil It's a good project, but surprised you find it legendary. Implementing TCP is a pretty standard project for an undergrad networking course, right?

Yeah I'm still thinking about this five years later. legend

that janestreet intern implemented tcp/ip

@lemire @lamchester @trav_downs Does your result hold if you add a built-in expect that the branch is not likely? Wonder if the generated code was an always-taken branch instead of never-taken

@lamchester @trav_downs @tlipcon I find the first part of this documentation intriguing. Empirically, I have found that adding a never taken branch made predictions worse. Not on AMD and I could have a methodological issue: lemire.me/blog/2019/11/0…

uarch experts: do not-taken forward branches (eg to exceptional paths never expected to be taken) pollute BTB? Is it better to transform the code to produce an invalid pointer load and catch it in a signal handler? (cc @trav_downs)

@trav_downs Thanks!

@tlipcon I thought this was explicit about never-taken branches not using resources, but it's not in there like that but rather implicit in the description of how it works. They also talk a bit about how always-taken branches can be handled to pollute less, but they still do pollute.

@spandan67776220 @ApacheKudu Nope. We implemented raft which I think has a spec

@tlipcon @ApacheKudu Reading paper on kudu. Does kudu have a tla+ spec ? Couldn’t find any.

Cool to see this blog from Meta eng on adapting the @ApacheKudu raft implementation to run their MySQL replication! engineering.fb.com/2023/05/16/dat…