Dev Tyagi

499 posts

Dev Tyagi banner
Dev Tyagi

Dev Tyagi

@devptyagi01

Inventing @Temple | ex - Software Engineer at @zomato @AppSecure @Amazon @TartanHQ

Gurgaon, Delhi, Ghaziabad 가입일 Nisan 2017
1.1K 팔로잉2.2K 팔로워
고정된 트윗
Dev Tyagi
Dev Tyagi@devptyagi01·
Last week, I faced an interesting problem at work. I received a critical warning from AWS regarding our PostgreSQL RDS cluster, it was approaching transaction ID wraparound, something that can cause a full database outage if not handled in time.⁣ ⁣ This was a good reminder of how important it is to understand the internals of the systems we rely on every day.⁣ ⁣ For those unfamiliar, PostgreSQL uses a 32-bit transaction counter (XID) that wraps around after ~2.1 billion transactions. If old rows aren’t vacuumed and frozen in time, PostgreSQL can no longer determine which rows are visible, forcing it to stop accepting new writes to prevent data corruption.⁣ ⁣ In our case, it turned out that autovacuum wasn’t keeping up due to high write throughput and a few misconfigured thresholds.⁣ ⁣ What helped:⁣ ⁣ 👉🏻 Identifying large tables with unvacuumed rows.⁣ 👉🏻 Manually running 𝖵𝖠𝖢𝖴𝖴𝖬.⁣ 👉🏻 Tuning autovacuum workers settings and vacuum cost limits.⁣ ⁣ It’s one of those things that often goes unnoticed until it becomes urgent. If you're running PostgreSQL on RDS (or self-managed), keep an eye on 𝖺𝗀𝖾(𝖽𝖺𝗍𝖿𝗋𝗈𝗓𝖾𝗇𝗑𝗂𝖽) and ensure autovacuum is running efficiently.⁣ ⁣ Invisible debt in infrastructure has a way of surfacing at the worst possible time. This one got our attention early, worth sharing in case it helps others avoid a painful surprise.
Dev Tyagi tweet media
English
4
0
35
6.4K
Arpit Bhayani
Arpit Bhayani@arpit_bhayani·
Joined Razorpay as Principal Engineer II :) From being a long-time customer to now building parts of the system - it's a full circle. Fintech is a new territory for me - time to get under the hood of how money actually moves. New domain, same guarantees - availability, correctness, performance - just with real money on the line.
Arpit Bhayani tweet media
English
550
64
7.2K
582.2K
Dev Tyagi
Dev Tyagi@devptyagi01·
We're hiring 🚀
Deepinder Goyal@deepigoyal

We're recruiting at @temple. At Temple, we are building the ultimate wearable for elite performance athletes. A device that measures what no other wearable in the world measures, with a level of precision that doesn't exist yet. To build it, we need people who are obsessive about both the craft and the category. Engineers who are also athletes. People who will wear what they build, and hate it until it's perfect. Roles we're hiring for: 🟠 Analog Systems Engineers, Electronics Design Engineers 🟠 Embedded Systems Engineers — low-level HW bring-up, embedded signal and image processing, embedded AI 🟠 Design and Validation Engineers — sensors, actuators, battery, antenna, optics 🟠 CMF Engineers, Adhesive Materials Engineers 🟠 Sensor Algorithms Engineers — estimation theory, sensor fusion 🟠 Deep Learning Engineers — ML model development for physiological metrics 🟠 Computational Neuroscientists 🟠 BCI Engineers — real-time EEG/EMG acquisition and processing 🟠 Neural Decoding Researchers — brain activity to semantic mapping 🟠 Computer Vision Engineers — facial microexpression, subvocal muscle detection 🟠 Neuroimaging ML Engineers — multimodal sensor fusion 🟠 Last but not the least, product managers who work through Figma without needing a designer to hold their hand Important – we are building for people who push their bodies to the edge. We want to be those people, not just serve them. So only people who take fitness seriously, and have body fat <16% (men) and 26% (women) should apply. If you're not there yet but will commit to getting there in three months, you can apply too; but you'll be on probation until you are. Write to build@temple.com with your core skill as the subject line. Come find your tribe.

English
2
1
12
1.3K
Dev Tyagi
Dev Tyagi@devptyagi01·
@arpit_bhayani I learned about it the hard way - faced it in production and all writes had stopped 🫡 x.com/devptyagi01/st…
Dev Tyagi@devptyagi01

Last week, I faced an interesting problem at work. I received a critical warning from AWS regarding our PostgreSQL RDS cluster, it was approaching transaction ID wraparound, something that can cause a full database outage if not handled in time.⁣ ⁣ This was a good reminder of how important it is to understand the internals of the systems we rely on every day.⁣ ⁣ For those unfamiliar, PostgreSQL uses a 32-bit transaction counter (XID) that wraps around after ~2.1 billion transactions. If old rows aren’t vacuumed and frozen in time, PostgreSQL can no longer determine which rows are visible, forcing it to stop accepting new writes to prevent data corruption.⁣ ⁣ In our case, it turned out that autovacuum wasn’t keeping up due to high write throughput and a few misconfigured thresholds.⁣ ⁣ What helped:⁣ ⁣ 👉🏻 Identifying large tables with unvacuumed rows.⁣ 👉🏻 Manually running 𝖵𝖠𝖢𝖴𝖴𝖬.⁣ 👉🏻 Tuning autovacuum workers settings and vacuum cost limits.⁣ ⁣ It’s one of those things that often goes unnoticed until it becomes urgent. If you're running PostgreSQL on RDS (or self-managed), keep an eye on 𝖺𝗀𝖾(𝖽𝖺𝗍𝖿𝗋𝗈𝗓𝖾𝗇𝗑𝗂𝖽) and ensure autovacuum is running efficiently.⁣ ⁣ Invisible debt in infrastructure has a way of surfacing at the worst possible time. This one got our attention early, worth sharing in case it helps others avoid a painful surprise.

English
0
0
4
1K
Arpit Bhayani
Arpit Bhayani@arpit_bhayani·
PostgreSQL is a fascinating database, and if you want to explore something super interesting in it, I would highly recommend reading about these five niche concepts 1. Transaction Wraparound Problem 2. Transaction ID Freezing 3. Integration of Freezing Logic in VACUUM 4. Aggressive Freezing vs. Lazy Freezing 5. Multi-page Freezing and Freeze Map Maintenance To give you all a gist, Freezing in PostgreSQL is all about replacing very old transaction IDs with a special value to avoid problems when 32-bit transaction IDs run out and start over. You will find some videos on the first two and will need to use your favorite LLM to explore the other three. It might take some time, but every single concept will blow your mind and make you curious about database internals. It is a fascinating rabbit hole.
English
19
60
1.1K
81.6K
Dev Tyagi
Dev Tyagi@devptyagi01·
Another #Hiring post 🚨 We're also looking for Backend Engineering Interns at @temple. Skillset preferred (again, not limited to): Go, Python, Kafka. Please note that we're only looking for people who're okay with working on-site. Location: Gurgaon, India. Do reach out with your Resumes / Portfolios / GitHub profiles, or tag someone you might know who can be a good fit.
English
97
14
357
28.6K
Dev Tyagi
Dev Tyagi@devptyagi01·
Update: Have received a lot of applications, going through all of them. Will take time.
Dev Tyagi@devptyagi01

#Hiring Post 🚨 We at @temple are looking for Frontend Engineering Interns. Someone with high agency - who enjoys complete ownership, and finds a fit in early-stage companies developing at a high pace. Skills preferred (but not limited to) - Typescript, React, Vue. Please reach out to me or tag someone you might know who would like to be a part of a new age health startup.

English
1
0
8
1.3K
Dev Tyagi
Dev Tyagi@devptyagi01·
I’ve received a lot of applications, Will review all later tonight.
English
2
0
6
2.2K
Dev Tyagi
Dev Tyagi@devptyagi01·
#Hiring Post 🚨 We at @temple are looking for Frontend Engineering Interns. Someone with high agency - who enjoys complete ownership, and finds a fit in early-stage companies developing at a high pace. Skills preferred (but not limited to) - Typescript, React, Vue. Please reach out to me or tag someone you might know who would like to be a part of a new age health startup.
English
158
19
616
55.7K
Dev Tyagi 리트윗함
Deepinder Goyal
Deepinder Goyal@deepigoyal·
Coming soon. Follow @temple for more updates.
Deepinder Goyal tweet media
English
674
208
5.1K
1.8M
Dev Tyagi
Dev Tyagi@devptyagi01·
The worst part about today's @Cloudflare outage was that the dashboard was inaccessible. There was no way we could go ahead and change the DNS records. And to make things worse, even our domain registrar’s site depended on Cloudflare… so changing nameservers wasn’t an option either.
English
1
1
17
1.7K
Dev Tyagi 리트윗함
Deepinder Goyal
Deepinder Goyal@deepigoyal·
I’m not sharing this as the CEO of Eternal, but as a fellow human, curious enough to follow a strange thread. A thread I can’t keep with myself any longer. It’s open-source, backed by science, and shared with you as part of our common quest for scientific progress on human longevity. Newton gave us a word for it. Einstein said it bends spacetime. I am saying gravity shortens lifespan.  Read on, and tell me what you think.
Deepinder Goyal tweet mediaDeepinder Goyal tweet mediaDeepinder Goyal tweet mediaDeepinder Goyal tweet media
English
944
919
7.5K
10.3M
Dev Tyagi 리트윗함
Deepinder Goyal
Deepinder Goyal@deepigoyal·
So many people keep asking me about Continue. What is it? What are you up to? Here you go... Continue started as a research effort two years ago, with the belief that if the human body is a system, it should also have its leverage points. The simple levers that, when adjusted, could fundamentally alter how we age and live. Along with a team of initially skeptical researchers, we've been investigating a penny-drop insight about human aging. Something that's been hiding in plain sight for eternity. But more on that in a few weeks. We are at the tail-end of the research on this hypothesis, which if true, could fundamentally change our understanding of biology and aging. But today isn't about that. It's bigger than that. Today, we are expanding Continue Research to include a $25 million fund (entirely personally backed) to support researchers across the world who dare to ask simpler questions than anyone else. Who believe biology might be far simpler than we've made it. For over a decade, I have believed that most of the world’s problems stem from our short human lifespans. Continue Research’s goal is to extend healthy human function long enough that humans stop making short-term decisions. Read more about Continue’s purpose at – continue.com This will be a multi-decadal journey. Our goal here is to become a small catalyst in humanity's journey of conscious evolution. To lead us into the Post-Darwin era. Researchers, please look us up and apply for funding/grants at – continue.com/researchers ——— PS: Continue is not a part of Eternal
English
253
351
3.3K
651.4K
Dev Tyagi
Dev Tyagi@devptyagi01·
Finally upgraded my setup, thanks to @IKEAIndia 🤌🏻 What else should I add? 👀
Dev Tyagi tweet media
English
2
0
8
662
Dev Tyagi
Dev Tyagi@devptyagi01·
@arpit_bhayani Hahahaha YES! Binged watched something after soooo long and died laughing! 😂 Perfect weekend away from Tech.
English
1
0
5
1.1K
Arpit Bhayani
Arpit Bhayani@arpit_bhayani·
The climax of The Ba***ds of Bollywood is 🔥 duniya haseeno ka mela will never be the same for me 😂
English
21
3
269
26.9K
George Stock
George Stock@georgesttock·
Nano banana + MakeUGC + Veo3 = Ad Factory This agent creates 100s of ads every day - No actors - No product - No $10k/month agencies You're able to clone your comepitors ads with AI Paste there ad -> Pick an avatar and regenerate. Comment "NANO" and I'll send you the agent + the full playbook (must be following)
George Stock tweet media
English
4.1K
554
5.7K
700K
Dev Tyagi
Dev Tyagi@devptyagi01·
Another day of digging interesting facts from open-source codebases! 💡 I just discovered how ClickHouse stores Arrays efficiently using values + offsets ClickHouse has a reputation for being ridiculously fast at analytical queries. Part of that speed comes from its columnar storage design, but one detail that surprised me is how it stores arrays and nested data structures. Instead of keeping each row’s array as is, ClickHouse explodes arrays into two separate columns: 👉🏻 Values column: a flat sequence of all elements from all arrays. 👉🏻 Offsets column: an array of integers marking where each row’s array ends. 📋 A Naive Approach (Why Not Just Store JSON?) Imagine you have a table with arrays: CREATE TABLE example ( id UInt32, tags Array(String) ) ENGINE = MergeTree(); And you insert some rows: id tags 1 [ "db", "metrics" ] 2 [ "ai", "ml", "genai" ] 3 [ "infra" ] A naive database might: 👉🏻 Store each row’s array as a blob (["db", "metrics"]), bad for compression. 👉🏻 Store each row in a separate subtable, bad for query speed. ✅ The ClickHouse Way: Exploding Arrays ClickHouse stores the same data in this format: tags.values = [ "db", "metrics", "ai", "ml", "genai", "infra" ] tags.offsets = [ 2, 5, 6 ] 👉🏻 tags.values holds all elements in sequence, concatenated across rows. 👉🏻 tags.offsets tells you how many elements belong to each row, by marking the ending index. So how do you reconstruct the original row arrays? ✓ Row 1: take values from index 0 to 1 (offset 2) → [ "db", "metrics" ] ✓ Row 2: take values from index 2 to 4 (offset 5) → [ "ai", "ml", "genai" ] ✓ Row 3: take values from index 5 to 5 (offset 6) → [ "infra" ] That’s it! Two flat columns, but the ability to map back to nested arrays. ✅ Why This Is Brilliant 1. Append-friendly Adding a new row is just appending more values and a new offset. No reshuffling required. 2. Cache-friendly scans Columnar queries (like arrayJoin(tags)) only need to scan the flat tags.values, which is contiguous in memory. 3. Compression wins Flat columns compress better than row-level blobs because similar data (like repeated tags) sit close together. 4. Vectorized execution Queries operate on contiguous memory regions, enabling SIMD acceleration and fast parallel scans. 📋 Attaching the exact source code for reference: github.com/ClickHouse/Cli… #DatabaseInternals #Clickhouse
English
1
0
10
556