Araon

Memory companies rn

@thesudeshdas This is more of a "thank you for your contribution and best of luck" - text that you get after they fire you from a job

@ara0n_ Matrix incoming?

I made an interactive blog post about how JPEG image compression works: sophielwang.com/blog/jpeg

wrote about it here https://araon[dot]space/projects/sensex-simulation-engine

Built a simulation engine that combines historical BSE SENSEX data with news sentiment to see how market narratives drive move

@TGUPJ Hey I've been following you for some time, and love the photos you take, what camera/phone do you use to take these. Both here or on your blog/YouTube

16 years on this dial and seiko lumibrite refuses to age

As a big fan of the pink flower blooms in Bengaluru, I try to click pictures of these trees wherever I spot them. In that journey, I got curious and did a little research and plotted it on a map. There are around 27k tabebuia rosea trees in Bangalore currently.

Be ungovernable

Lessons from getting absolutely owned by SQLite: 1. Reads scale, writes don't (single writer reality check) 2. database/sql gives zero fucks about file-level locking 3. ab -c 20 reveals what unit tests politely hide 4. Mutex = quick band-aid, not architecture 5. Production? Move to Postgres yesterday Still love SQLite for side projects though. Just not for pretend scale. Which db hurt you the most so far? 😭

Read more at vmfunc.re/blog/persona

>be me, internet security nerd >hear about “Persona” identity verification thing used by big tech >decide to poke around infrastructure >Shodan finds random server with weird hostname: openai-watchlistdb.withpersona.com >lolwut.jpg >that’s not normal verify API >server not even behind Cloudflare like the legit stuff >download 53MB of source code that was accidentally exposed >No hack needed, just public source maps >inside code: > facial recognition that checks your selfie > compares you to “politically exposed persons” > 269 different verification checks > biometric face database stored for 3 years > can screen crypto wallets against sanctioned addresses > files SARs (Suspicious Activity Reports) directly to FinCEN > government deployment also exists under withpersona-gov.com > that’s a FedRAMP authorized system used by federal agencies > code shows admin panels, dashboards, device fingerprinting etc. > also finds another subdomain “onyx.withpersona-gov.com” > matches name of US ICE surveillance product ONYX > nobody knows why it’s there > public timeline shows this all ran since Nov 2023 > OpenAI didn’t disclose identity screening until much later

It's that time of the year!

Hi @TheOfficialSBI, please improve your Account Aggregator services. Thanks! cc @sahamati

@nirudhuuu Hi, can you share the read along streams

i’ve been studying both of these through read-along streams for a long time now. choosing computer science has been one of the best decisions I’ve made, especially because it introduced me to databases next, lld and redis will be the set of topics i’ll be self-learning

Claude pls, I have rent to pay 😭

@foldmoney_ Can I know when I registered too?

Love seeing the OGs drop by

@nirudhuuu "the complexity was pushed back into the application layer" - isn't this counter intuitive? Well most try to push much of the data operation into pg

nothing is better than spending an evening reading an engineering blog; today it was from the openai team on how they scaled postgresql what stood out was how methodical everything was. to start with, read request traffic was pushed to replicas wherever possible, but reads that were part of write transactions stayed on the primary leader and were aggressively optimized. by avoiding complex multi-table joins (at one peak they noticed a 12-table join), the complexity was pushed back into the application layer a lot of the underperforming queries came from ORMs that we usually assume are optimized out of the box. the takeaway is to build a habit of inspecting the SQL being generated instead of trusting abstractions blindly since they were maintaining a single-writer instance, the problem was handled pragmatically. some critical paths were read-only, so they were designed to survive even if the primary went down. the primary itself runs in high availability mode, with a hot standby always ready to take over in case of failure or migration a few noisy workloads were isolated by splitting traffic into buckets of high and low-priority. this way, a viral feature launch does not slow everything else down and the system keeps serving critical demand connection storms are inevitable at this scale, especially for an 800 million user base. they solved this with pgbouncer. connection pooling dropped setup time from ~50ms to ~5ms and prevented idle connections from piling up in memory cache misses are common when a storm of new requests comes in and they handled this with cache locking and leasing. only a single reader fetches data for a missed key and repopulates the cache, while the rest wait instead of hammering the database. that part was particularly impressive to me rate limits exist at every layer to stop retry spikes and expensive queries. schema changes are treated as production hazards (just like i do). only lightweight changes are allowed, with strict timeouts enforced, and backfilling tasks are rate-limited as well this tells me databases will not magically scale themselves. rather, when engineering teams measure and monitor p95 and p99 metrics carefully and maintain query discipline, the system can meet the demand

@nirudhuuu I'm more of a yeppie guy

principal engineer btw