Are your Postgres servers melting down right now? Congratulations AND you need PG Dog - the founder Lev is one of the world's experts at scaling and sharding databases - he did it for Instacart and his software can do it for you.
If your replicas are falling behind and your IOPS are pegged, it's time.
pgdog.dev
hot take: databases should be single-threaded
someone a long time ago decided that database shards should be multi-threaded. ever since then, we've had to worry about transactions, serializability, race conditions, and locking.
instead, we should have single-threaded shards, and shard aggressively and early.
we switched from servers to serverless by building apps with statelessness in mind. it's time to switch from multi-threaded DBs to sharded single-threaded DBs by building apps with sharding in mind.
many will say that you can't JOIN across shards if you do this. but if you're JOINing across shards on your *writer* node you're already doing something wrong in the current world. your read replicas will still be able to JOIN just fine.
that way, scale comes naturally with no extra effort.
sharding sounds hard but is it really harder than worrying about transaction guarantees with every query you do?
(yes, i'm serious. at the very least i think this type of database is extremely underexplored.)
Databases aren't greenfield projects. I've been working on this feature for a while: we can re-shard Postgres online, terabytes at a time.
pgdog.dev/blog/sharding-…
Made a video about load balancing in PgDog. You can get 99.99% availability with just plain old Postgres and horizontal scaling for your read queries...without changing a line of code in your app.
youtu.be/ZaCy_FPjfFI?si…