When building any database, writing the core engine is only half the battle. If you can't observe what your system is doing under load, you are in for a surprise, and that too a bad one.
To gain this transparency and reliability, two components are critical (for in-mem databases) - deterministic telemetry and an eviction mechanism to keep memory usage bounded.
And, I just published a video where we implement the Redis's INFO command for real-time statistics and dive into the allkeys-random eviction strategy.
This is the 14th video in the Redis Internals series. We look at how Redis implements its point-in-time statistics, and we mirror that exact protocol so that we can hook our engine directly into existing Prometheus and Grafana toolchains.
We then take those observability metrics and watch our eviction logic in action. By utilizing Go's native hash table iteration behavior, we achieve a pseudorandom cleanup mechanism that caps our database size safely under load.
If you have ever wondered how telemetry protocols are designed to be both human-readable and scrapable, or how Redis does random-sampling-based eviction, give this video a watch. 14 videos are now live:
1. Why Single-Threaded Redis Is Fast
2. Writing a TCP Echo Server
3. Wire Protocols
4. Implementing RESP
5. Implementing PING
6. Understanding Event Loops
7. Implementing Event Loops
8. Implementing GET, SET, and TTL
9. Implementing DEL, EXPIRE, and Cleanup
10. Key Eviction Strats and Implementing First Eviction
11. Implementing Command Pipelining
12. Implementing AOF Persistence
13. Objects, Encodings, and Implementing INCR
14. Implementing INFO and allkeys-random Eviction
Hope this helps you better understand database internals and spark that engineering curiosity.
Give it a watch
English
