Clockwork Labs
119 posts
@clockwork_labs
Creating beautiful machines. Currently working on https://t.co/rUQoxP1R8d and https://t.co/ZIpXxgJDMZ
Most database benchmarks are bad. The problems are often subtle. Mismatched instances, cold caches, and coordinated omission all produce misleading results. Today, we’re publishing guidance on how to do it right, and updating our acceptable use policy to allow benchmarking.
vibecoder asks claude code to build a chat app, gets a working prototype in 20 minutes, immediately tweets "just killed slack and discord"… brother you don't even know what a distributed system is. you don't know what database replication means. you have no idea how websocket connections behave at scale or what happens when 50k people are online at once and someone's message needs to show up in 200ms across 3 continents slack has engineers making $300k+ who have spent a decade solving problems you don't even know exist yet. race conditions, eventual consistency, message ordering, presence systems, file storage at scale, search indexing across billions of messages your app works on localhost with 2 connections. that's not the same thing as "killing slack" that's a college homework assignment the prototype is maybe 0.5% of what makes these products actually work in production. the remaining 99.5% is infrastructure, reliability, edge cases, and years of iteration on problems that only surface when real humans use your thing at scale and the worst part is the confidence. "yeah its not perfect but ai one-shotted it, just need to adjust a few things and deploy" - the few things you need to adjust IS the entire product. thats like pouring a foundation and saying you basically built a skyscraper, just need to adjust a few things ai is genuinely incredible for building tools and prototypes. i use it every day. but there's this weird thing happening where people who have never shipped anything to real users at scale now think the hard part of software is writing the first 200 lines of code it never was bro
Sharded Postgres is closer than you think. Video conferencing on Neki. 720p. 20fps. Cross-shard queries. Built by @__spongeboi
Earlier this week, SpacetimeDB launched v2 of their product with a weird marketing video. It was full of memes, and mocking other databases; their CEO showed up drinking "competitor's tears". I found it off-putting, but it did pique my interest enough to do a technical review of their product. In summary: it's not a very good database, and it is a big marketing miss. Database marketing is hard! They had a good chance to launch this and make people go "Wow! It's like Redis on steroids! You can build a whole MMORPG on top of this!". But instead they invited some very awkward comparisons with their benchmarks. Any technical person who looked at their offering probably thought: "Wow, 190k QPS in a relational database? I wonder what's the catch". The catch is that it's pretty much just an in-memory hash table with a single lock in front of it. Some interesting ideas, some big footguns. But that's pretty much it. Here's my full technical review, including some thoughts on database marketing, if you're interested 👇
This is a dangerous and fundamental misunderstanding of how SpacetimeDB works. By default, SpacetimeDB waits for all data to be durably persisted to disk BEFORE exposing it to clients. The window in which you can lose data not "very small", it's absolutely 0. SpacetimeDB can be configured to expose data to clients before data is persisted, but so can Postgres (synchronous_commit = off). The genius of SpacetimeDB is that although we fsync asynchronously, we buffer all client reads (and write acks) until we've guaranteed they are persisted to disk. Not only does this guarantee durability of transactions from the perspective of clients, it also allows us to decouple fsync latency and fsync throughput. Pipelining is a powerful optimization. @mihir20/understanding-synchronous-commit-in-postgresql-54cb5609a221" target="_blank" rel="nofollow noopener">medium.com/@mihir20/under…
That being said, it's understandable how people who spent their careers just operating Postgres instead of designing their own database engine could be so confused."The WAL is asynchronous, and is flushed periodically to disk on the background (by default, every 50ms)." these kinds optimizations are appealing because the "window" at which you could lose data is very small. usually, though, production workloads are continuously writing data. when continuously writing, you're actually *guaranteed* to lose *some* data each time your host loses power. this is generally not desirable.
