Data Engineering Weekly

More ETL pipelines will run next year than ever before. And ETL is still dead. Not dead, like nobody uses it. Dead like landlines — they work, but nobody builds their strategy around one. dataengineeringweekly.com/p/etl-is-dead

As we move from dashboards to autonomous agents, something breaks. Systems of record capture what happened, not why. Why data platforms need Truth Registries + Context Graphs for the agentic era 👇 dataengineeringweekly.com/p/the-missing-… #DataEngineering #AgenticAI #Graphs #LLMs

Data Engineering Weekly's 254th edition is out. Context Graph is the new talk of the town!!

Data Contract: There was no shortage of activity around the topic. Definitions were proposed and refined. Conceptual boundaries were drawn and redrawn. I pen down a reflection of the Data Contracts here dataengineeringweekly.com/p/data-contrac…

This week, it is mostly about Multi-Agent Architecture. Do you think the data infrastructure is ready for a multi-agent architecture? Where is the gap? dataengineeringweekly.com/p/data-enginee…

Is semantic Spec Good enough to run an enterprise system? I listed challenges to adopting the Iceberg Rest Catalog dataengineeringweekly.com/p/a-critique-o…

Continuing our yearly tradition of Year in Review Data Engineering Weekly, we published the 2025 Year in Review. What do you think is the most notable trend of 2025? dataengineeringweekly.com/p/dew-the-year…

LinkedIn moves FishDB to Rust, DoorDash builds AI swarms, and Dropbox masters context engineering. 🤯 Data Engineering Weekly #247 is packed with system design deep dives from the best engineering teams. dataengineeringweekly.com/p/data-enginee…

This is the most personal essay that I have written in Data Engineering Weekly. I shared a few key moments in my life and how fortunate I was to meet mentors along my professional journey, which shaped my career. dataengineeringweekly.com/p/thinking-lik…

🚀 Data Vault vs. Dimensional Modeling vs. Medallion Architecture — When viewed through a modern enterprise data lens, these techniques interlock. I break down how in Part 2 of my “Revisiting the Medallion Architecture” series. dataengineeringweekly.com/p/revisiting-m…

As a data engineer, you can't treat zero-party (consent) and third-party (inferred) data the same way. This distinction is critical for building systems that are scalable, private, and trustworthy. Here’s my guide: dataengineeringweekly.com/p/engineering-…

I wrote my thoughts on Supporting Our AI Overlords. dataengineeringweekly.com/p/what-support…

The 238th edition of Data Engineering Weekly is available, featuring exciting Data & AI articles. Read more: dataengineeringweekly.com/p/data-enginee…

For years, I've been a skeptic of the Medallion Architecture naming convention. However, despite my initial reservations about the naming, I have come to appreciate the value of a bounded definition. I've shared my thoughts on Medallion Architecture. dataengineeringweekly.com/p/revisiting-m…

The 234th edition is out with the latest trends and thoughts in Data Engineering.

I just apologized to my Claude Code. It truly deserves sympathy—stuck untangling a Jackson dependency issue that no AI should ever have to endure. Some battles are timeless, and Jackson always finds a way to remind us.

The 233rd edition of Data Engineering Weekly is out. dataengineeringweekly.com/p/data-enginee…

Dimensional Modeling or data vault, which one is better? Property graph vs RDF, which one is best for metadata? All these techniques are defined for human-in-the-loop. The goal of a DW is to provide accurate metrics in a timely fashion. Do we still model the data for humans?

🎙️ New DEW pod: Bauplan CTO Jacopo Tagliabue on function-native pipelines, immutable versioning, Arrow-powered speed, and why vertical > horizontal for most 10–100 GB jobs. Apple:podcasts.apple.com/us/podcast/ins… Spotify:open.spotify.com/episode/5xebhI… Substack:dataengineeringweekly.com/p/insights-fro…