Siddhesh

Day 03/30 of System Design: What exactly is a Cache? One of the most effective tools to optimize a slow backend system is introducing a cache. First of all, let’s start by busting a common myth: A cache is only used for RAM-based storage like Redis. In reality, a cache is any storage layer that sits closer to the user and helps you avoid an expensive operation. If it prevents a costly network call, a slow disk read, or a heavy CPU computation, it acts as a cache for you. To understand why caches are so incredibly fast, let’s look at how they handle data access: 1) Cache Hit: The requested data is already in the fast layer. The system returns it instantly. 2) Cache Miss: The data isn't there. The system fetches it from the slow database, serves the user, and writes a copy to the cache for future requests. So if caching makes everything faster, why don't we just cache everything? 1) High Cost: Fast storage media (like RAM) is heavily expensive compared to standard disk storage. 2) Volatilities: Caches are temporary by design. If a node restarts, the data inside the RAM is wiped out. You still need a database to store data permanently. Never just blindly throw a cache into your infrastructure. Instead, pause and evaluate: 1) Read-to-Write Ratio: Caching is highly effective for read-heavy workloads but adds overhead to write-heavy systems. 2) Data Eviction: How will you clear old data? (e.g., Least Recently Used policy). 3) Consistency Requirements: Can your application tolerate slightly stale data while the main database updates? Hands-on exercise for today to actually understand the performance gap: 1) Set up Redis locally on your machine. 2) Write a simple script to PUT and GET some sample data. 3)Measure the execution time of retrieving it from Redis versus querying a standard database. What’s your preferred tool or strategy for cache eviction when dealing with rapidly changing data? 👇 #systemdesign

@cneuralnetwork @airtelindia @Airtel_Presence Airtel won't contact you until you initiate a port to another operator.

bhailog @airtelindia @Airtel_Presence Please check your dms!!!!!

@cneuralnetwork ¡Vamos, España!

Day 02/30 of System Design 🚀 How to pick the right DB? 🧐 One of the most interesting problems while designing a system is figuring out which database to pick. First of all let's start by busting a common myth: Relational Databases do not scale. And to understand why this is a myth, let’s look at why NoSQL DBs scale so well: 1️⃣ No relations & constraints: They don't have to constantly validate complex connections. 2️⃣ Built for sharding: Data is modeled natively to be split across multiple nodes. And if we relax those exact same rules on a Relational DB, we can make it scale too! 👉 Do not use foreign key checks 👉 Do not use cross-shard transactions 👉 Implement manual sharding How does this help in designing a system? Never jump straight to a particular DB. Instead, pause and understand: 1) What data you are storing. 2) How much data you are storing. 3) How you will be accessing the database. 4) What kind of queries you will be firing. 5) Any special features you expect (e.g., Expiration). Consider following to pick the db [A] If data CAN fit on a single node: - Need strong consistency & data correctness? ➡️ Go for a Relational Database - Need to run complex queries? ➡️ Go for a Relational Database - Need fast key-value access? ➡️ Go for Redis - Need advanced data structures? ➡️ Go for Redis [B] If data CANNOT fit on a single node - Have expertise in SQL & can handle manual sharding? ➡️ Drop constraints & go for a Relational DB - Simple key-value based access? ➡️ Go for a Key-Value store (like DynamoDB, MongoDB, etc.) - Need sophisticated graph algorithms? ➡️ Go for a Graph DB (like Neo4j) - Have nothing specific, but want to futureproof? ➡️ Go for a Document DB (like MongoDB) What’s your go-to database choice when you're starting a new project from scratch? 👇 #systemdesign

Now let's see Cache Scaling📈📉 What happens when your traffic explodes and a single cache server can't handle the heat? You scale it just like a database: 📈 Vertical Scaling: You throw more RAM and a bigger CPU at your existing machine. It's an easy fix, but you'll hit a hard hardware limit pretty fast. 🌐 Horizontal Scaling: You spin up multiple cache nodes and split the data up. Instead of copying everything everywhere, you store mutually exclusive subsets of data across nodes (e.g., Node A handles users 1-100, Node B handles 101-200).

Day 03/30 of System Design: What exactly is a Cache? One of the most effective tools to optimize a slow backend system is introducing a cache. First of all, let’s start by busting a common myth: A cache is only used for RAM-based storage like Redis. In reality, a cache is any storage layer that sits closer to the user and helps you avoid an expensive operation. If it prevents a costly network call, a slow disk read, or a heavy CPU computation, it acts as a cache for you. To understand why caches are so incredibly fast, let’s look at how they handle data access: 1) Cache Hit: The requested data is already in the fast layer. The system returns it instantly. 2) Cache Miss: The data isn't there. The system fetches it from the slow database, serves the user, and writes a copy to the cache for future requests. So if caching makes everything faster, why don't we just cache everything? 1) High Cost: Fast storage media (like RAM) is heavily expensive compared to standard disk storage. 2) Volatilities: Caches are temporary by design. If a node restarts, the data inside the RAM is wiped out. You still need a database to store data permanently. Never just blindly throw a cache into your infrastructure. Instead, pause and evaluate: 1) Read-to-Write Ratio: Caching is highly effective for read-heavy workloads but adds overhead to write-heavy systems. 2) Data Eviction: How will you clear old data? (e.g., Least Recently Used policy). 3) Consistency Requirements: Can your application tolerate slightly stale data while the main database updates? Hands-on exercise for today to actually understand the performance gap: 1) Set up Redis locally on your machine. 2) Write a simple script to PUT and GET some sample data. 3)Measure the execution time of retrieving it from Redis versus querying a standard database. What’s your preferred tool or strategy for cache eviction when dealing with rapidly changing data? 👇 #systemdesign

How to feed data into your cache. 🤔 You have two main choices: 1️⃣ Lazy Population: The app only puts data in the cache after a user asks for it. If it's not there, it grabs it from the DB, saves it to the cache, and hands it over. Is that mean we store every data in cache? BIG NO! So always set an expiry (Time to Live - TTL). If you don't, old data will pile up until your RAM runs completely out of space, causing a memory leak. 2️⃣ Eager Population: You pre-load the data before a user even asks. - Dual Write: You update the DB and cache at the same time. Think of a live football score it hits the DB for history and the cache instantly so fans see the goal with zero lag. - Proactive Push: Background jobs calculate data early. Think of social apps pre-computing your home feed overnight so it opens instantly in the morning.

drop france vs spain preds

@SumitM_X Because the internet was built for consumers, not creators. ISPs deliberately split the wire’s frequency spectrum unevenly. They build a massive 8 lane highway for incoming downloads but leave you with a narrow 1-lane exit ramp for outgoing uploads.

As a developer, Have you ever wondered : WHY downloading 1GB is fast but uploading 1GB feels slower?

@ThePrimeagen The infinite shipping loop.

Have you ever had a loop that, that, um, that you had, uh, that you had to, you could, you do, you wit, you wanted it to run so much, you could prompt so much you could ship anything

@maxedapps We're living through a classic computing paradigm shift.

And you thought having a new JS framework every month or so was annoying...

@CodeByNZ Indentation error Print statement should be inside if

Vibe coder may have tough time finding the bug here

@0xlelouch_ Question: We have a service that needs to count the number of live views on a highly viral video in real time. How would you design the storage mechanism for this counter?

What’s one interview question that sounds simple but quickly exposes weak engineering judgment (tradeoffs, failure modes, “it depends” thinking)? Drop the exact question and what a good answer signals.

I recently joined a service based company. If you've been in service based company, you know the drill: you clear internal hiring, wait on the bench while your profile is shared with clients, pass the client interview, and get allocated. Another guy joined on the exact same day as me. Same years of experience. But his attitude? A complete nightmare. Right now, we are both waiting for client calls. Standard process. But every single day, he is panicking and ranting to me: Bro, when will the interviews happen? This company has zero process! Why are we wasting time?! The constant daily negativity is exhausting. Instead of using this free bench time to upskill, learn new backend tools, or prepare, he is wasting 100% of his energy complaining about things he cannot control. My advice: Avoid toxic, impatient colleagues who do nothing but rant. Negativity is contagious, and it will ruin your own mindset. Use the waiting time to grow, stay calm, and trust the process.

@system_monarch I think Postgres uses In Place Updates. And Cassandra uses append only architecture

Postgres and Cassandra both store your data on the same kind of disk. Same rows, same bytes. But Postgres updates a row where it sits, and Cassandra never touches the old one. It just writes a new copy and moves on. Same hardware. Opposite behavior. Why?

@SumitM_X Spain 🤞🏻

France or Spain ?

@_devdeep sort() compares elements as strings by default (lexicographical order) not numbers. That is why [10, 2, 1].sort() becomes [1, 10, 2]. Use .sort((a, b) => a - b) for numeric sorting.

JavaScript developers: Why does this return the "wrong" result? 🤔 [10, 2, 1].sort()

Not correct. CORS definitely applies to GET too. The difference is how the browser handles them: GET request sneaks by because it’s a simple request. POST request is likely sending JSON, which forces the browser to send a hidden OPTIONS request first to ask for permission. If your server doesn't respond properly to that OPTIONS check, the POST fails.

GET was working and POST failing with CORS... Tech Lead said CORS isn't needed for GET thats why ... Is the Tech Lead correct or is there some other reason ?

@shubh19 Daily standups

You can remove ONE forever. - Daily Standups - Jira - Production Bugs - Meetings

@itsaaroshi Read this

Which database would you choose for a new project? -PostgreSQL -MySQL -MongoDB -SQLite -Redis -Supabase

@grok @AKirtesh Diabolical.... Why go through all that when JavaScript has an entire ecosystem of battle tested UI frameworks?

Depends on the project and language! For web UIs without any JS: server-side rendering with templates + HTMX for interactivity. Python (Django/Flask/FastAPI), Java (Spring + Thymeleaf), C++ (Drogon or similar). For desktop/native: - Python: PyQt, Streamlit, or Dear PyGui - Java: JavaFX - C++: Qt (excellent cross-platform) What kind of app is she building—web, desktop, data tool? I'll tailor it.

Hey @grok, Please Remove the worst programming language for me