Data Engineer Things

May 2026 Data Engineering Newsletter is out 🚀 🔹 1,101 DEs surveyed on AI adoption: the gap is real 🔹 Spotify saved 10 engg weeks with coding agents 🔹 Meta mapped tribal knowledge across 4,100+ pipeline files with AI & more open.substack.com/pub/dataengine… #DataEngineering #AgenticAI

This month in data engineering, April 2026: Context engineering for AI-native platforms, Spotify's pipelines behind Wrapped 2025, Uber's Petastorm for high throughput, DuckLake 1.0 production release, and more. All packed into the latest DET Newsletter: dataengineerthings.substack.com/p/data-enginee…

This month in data engineering, March 2026: OpenAI's P99 latency with 800 million users, Netflix's LLM post-training, LinkedIn's exabyte-scale clusters, ETL → ECL, and more. All packed into the latest DET Newsletter: dataengineerthings.substack.com/p/data-enginee…

February 2026 DET Newsletter is live! Netflix's Data Bridge — unified data movement Uber's trillion-record lake — Hudi at scale LinkedIn's job ingestion — millions of records AI skills CLI for DE workflows Read it here open.substack.com/pub/dataengine… #DataEngineering #ApacheHudi #AI

This month in data engineering, Jan 2026: Templatizing Spark Declarative Pipelines, Meta’s Video Invisible Watermarking at Scale, Lyft’s Feature Store architecture and more, all of which is packed into the latest DET Newsletter. open.substack.com/pub/dataengine… #dataengineering #AI

This month in data engineering, December 2025: Thinking like a Data engineer, Real-time distributed graph, Secret behind super fast databases, Multimodal data workloads, and more. All packed into the latest DET Newsletter: dataengineerthings.substack.com/p/data-enginee… #dataengineering #AI

This month in data engineering, Nov 2025 🚀: how TikTok built a unified lakehouse using Apache Paimon to support large-scale recommendation models, processing 600TB in 5 hours with Flink CDC for real-time features, lessons from experienced DEs and more: open.substack.com/pub/dataengine…

This month in data engineering, October 2025: Netflix's journey to trillion-row scale. Open-source stream processing in Rust. Are platforms really simplifying our lives? Tips on cutting through architectural complexity. Read the latest DET Newsletter: buff.ly/rJEc01e

This month in data engineering, Sept 2025: Open-Source Breakthroughs with Sail and MLflow, Hidden Infrastructure Behind ChatGPT, Streaming Data Into Iceberg, Data Quality Tips, and More. All packed into the latest DET Newsletter: dataengineerthings.substack.com/p/data-enginee… #dataengineering #AI

We are looking for speakers for our Warsaw Meetup (next on Sept 25). All topics within the broad scope of data engineering are welcome. To speak, submit the CFP form: meetup.dataengineerthings.org/cfp To attend, join the meetup group: meetup.dataengineerthings.org/warsaw #dataengineering

This month in data engineering, August 2025: Agentic Data Engineering, NY Meetup on September 25, Data Modeling for Data Products, What Makes a Great Data Engineer, and more. All packed into the latest DET Newsletter: dataengineerthings.substack.com/p/data-enginee… #dataengineering #AI

We are looking for speakers for Data Engineer Things (DET) Meetups in several cities: Bay Area (early Sept), NYC (Sept 25), and Warsaw (late Sept). Submit talk ideas here: meetup.dataengineerthings.org/cfp #dataengineering

This month in data engineering, July 2025: new AI tools, open-source highlights from DAIS 2025, Seattle Meetup on July 24, Git-for-Data, practical DE tips, and more. All packed into the latest DET Newsletter: dataengineerthings.substack.com/p/data-enginee… #dataengineering

The Data Engineer Things (DET) Medium publication has reached 200K monthly views and 30K followers! This milestone isn't possible without our writers and editors. And we’re just getting started! We’d love for you to write with us. Apply to be a writer: buff.ly/Z0TyxAU