Delta Lake

📣 Join us for the next Delta Lake community meetup on May 19! Agenda: 🔹 Issues & PR backlog 🔹 Unified Kernel 🔹Community highlights Register: luma.com/deltalake-0519 #DeltaLake #OpenSource #Rust #OpenLakehouse

[Ep 1] Open Lakehouse + AI: The Catalog Layer, Interoperability & AI-Agent Governance Learn how composable lakehouses center the catalog for metadata, versioning, and commit coordination. 👇 🔸 Interop: Iceberg REST, Unity Catalog (OSS), Spark, Delta Lake, Delta-RS, Iceberg. 🔸 Governance: credential vending, row/col filters, column masks, audits, and why “no governance was the easiest governance.” 🔸 Agents as data customers (Temporal). Fine-grained access beats blanket credentials on 24/7 workloads. 🎥 Full episode: youtube.com/watch?v=dEFAkS… #OpenLakehouse #UnityCatalog @ApacheSpark @ApacheIceberg @DeltaLakeOSS

Delta Lake 4.2 pushes Delta Kernel further (new Flink connector, richer types) and hardens catalog-managed tables (atomic ops, SQL evolution, UniForm). 👇 🔹 Kernel / Flink — New Kernel Flink connector (experimental): catalog-managed, catalog-coordinated writes, exactly-once. Replaces legacy connector deprecated in Delta Lake 4.0. 🔹 Schema (SQL) — INSERT INTO … BY NAME + autoMerge adds columns in-commit. delta.stats.skipping.forceOptimizeStatsCollection → data skipping on new columns without OPTIMIZE. 🔹 Types — Geospatial, Collation, Variant (shredding out of preview); Spark: full Variant schema conversion. 🔹 Catalog — RTAS / dynamic partition overwrite: one atomic commit. UniForm: Iceberg metadata at commit (reads immediate). HMS UniForm deprecated. 🔗 Read more → delta.io/blog/2026-04-1… #DeltaLake #OpenSource

The next evolution of Delta - Catalog-Managed Tables: delta.io/blog/2026-02-0…

Delta Lake Community Meetup [April 2026] x.com/i/broadcasts/1…

Iceberg Single File Commits: s.apache.org/iceberg-single…

Hey all! Thanks for joining us. 👏 Where are you tuning in from?

Here is a breakdown of what’s new: 🌟 [𝗦𝗽𝗮𝗿𝗸] 𝗨𝗻𝗶𝘁𝘆 𝗖𝗮𝘁𝗮𝗹𝗼𝗴 𝗠𝗮𝗻𝗮𝗴𝗲𝗱 𝗧𝗮𝗯𝗹𝗲 𝗲𝗻𝗵𝗮𝗻𝗰𝗲𝗺𝗲𝗻𝘁𝘀: REPLACE TABLE / RTAS and Dynamic Partition Overwrite support, automatic table schema/properties sync to catalog on table creation. 🌟 [𝗦𝗽𝗮𝗿𝗸] 𝗗𝗲𝗹𝘁𝗮 𝗦𝗽𝗮𝗿𝗸 𝗩𝟮 𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗼𝗿 - 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗿𝗲𝗮𝗱 (𝗲𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹): enhance streaming read capabilities for catalog-managed table by supporting critical options like startingTimestamp and skipChangeCommits. 🌟 [𝗙𝗹𝗶𝗻𝗸] 𝗡𝗲𝘄 𝗞𝗲𝗿𝗻𝗲𝗹-𝗯𝗮𝘀𝗲𝗱 𝗙𝗹𝗶𝗻𝗸 𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗼𝗿 (𝗲𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹): a brand-new Kernel-based delta-flink connector that enables Apache Flink to read, write, and interact with catalog-managed Delta tables. 🌟 [𝗞𝗲𝗿𝗻𝗲𝗹] 𝗚𝗲𝗼𝘀𝗽𝗮𝘁𝗶𝗮𝗹, 𝗩𝗮𝗿𝗶𝗮𝗻𝘁 𝗚𝗔, 𝗮𝗻𝗱 𝗖𝗼𝗹𝗹𝗮𝘁𝗶𝗼𝗻𝘀 𝘁𝗮𝗯𝗹𝗲 𝗳𝗲𝗮𝘁𝘂𝗿𝗲: Delta Kernel can now read and write tables using geometry/geography types with bounding-box data skipping, generally available Variant columns, and collated string types. 🌟 [𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆] 𝗩𝗲𝗿𝘀𝗶𝗼𝗻 𝗵𝗮𝗿𝗱𝗲𝗻𝗶𝗻𝗴 𝗳𝗶𝘅: The Delta project has undergone a substantial hardening effort across multiple surface areas, including stronger validation and dependency security scanning to proactively reduce supply-chain risk.

Delta Lake 4.2.0 is now available! 🚀 Dive into the full release notes here: 👉github.com/delta-io/delta… Huge thanks to everyone in the Delta community who made this release possible! 🎉 #DeltaLake #OpenSource #ApacheSpark #ApacheFlink

Join us for this first Delta Lake Community Meetup of 2026 on Tuesday, April 21 at 9AM PT! 🚀 We’re bringing the community together for a deep dive into the ecosystem, infrastructure enhancements, and the future project roadmap. Come get your technical questions answered live by the maintainers. ​ What we'll cover: 🔹 Latest Delta Lake updates and how the community is evolving 🔹 A technical look at infrastructure enhancements 🔹 The future of Delta Lake: Roadmap insights and a deep dive into Iceberg v4 compatible metadata 🔹 Live Q&A with the community RSVP ➡️ luma.com/deltalake-0426 #opensource #oss #deltalake #community

Delta Lake for Rust Hacking x.com/i/broadcasts/1…

𝗗𝗲𝗹𝘁𝗮 𝗟𝗮𝗸𝗲 𝟰.𝟭.𝟬 introduces significant new features, performance improvements, and critical platform upgrades, including full support for Apache Spark 4.1.0 and enhanced storage management in Unity Catalog. 🚀 What's inside Delta Lake 4.1.0? 🔹 𝗖𝗮𝘁𝗮𝗹𝗼𝗴-𝗠𝗮𝗻𝗮𝗴𝗲𝗱 𝗧𝗮𝗯𝗹𝗲 𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗺𝗲𝗻𝘁𝘀 🔹 𝗔𝗽𝗮𝗰𝗵𝗲 𝗦𝗽𝗮𝗿𝗸 𝟰.𝟭.𝟬 𝗦𝘂𝗽𝗽𝗼𝗿𝘁 🔹 𝗦𝗲𝗿𝘃𝗲𝗿-𝗦𝗶𝗱𝗲 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴 (𝗣𝗿𝗲𝘃𝗶𝗲𝘄) 🔹 𝗔𝗪𝗦 𝗦𝘁𝗼𝗿𝗮𝗴𝗲 𝗖𝗿𝗲𝗱𝗲𝗻𝘁𝗶𝗮𝗹𝘀 & 𝗘𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝗟𝗼𝗰𝗮𝘁𝗶𝗼𝗻𝘀 🔹 𝗔𝘁𝗼𝗺𝗶𝗰 𝗖𝗧𝗔𝗦 𝗳𝗼𝗿 𝗨𝗖 𝗧𝗮𝗯𝗹𝗲𝘀 🔹 𝗖𝗼𝗻𝗳𝗹𝗶𝗰𝘁-𝗙𝗿𝗲𝗲 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗘𝗻𝗮𝗯𝗹𝗲𝗺𝗲𝗻𝘁 Take a closer look at what’s new 👉 delta.io/blog/2026-03-0… #DeltaLake #OpenSource #ApacheSpark #UnityCatalog

Last call for questions!

Have questions? Drop them in the chat below! 👇

Best of Both Worlds: Leveraging delta-kernel-rs to Unify the Open Lakehouse x.com/i/broadcasts/1…

📣 𝗡𝗲𝘅𝘁 𝗢𝗽𝗲𝗻 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 + 𝗔𝗜 𝗪𝗲𝗯𝗶𝗻𝗮𝗿: 𝗧𝘂𝗲𝘀𝗱𝗮𝘆, 𝗠𝗮𝗿𝗰𝗵 𝟭𝟬! Open table formats promise engine-agnostic access, but independent protocol maintenance is costly. The Delta Kernel solves this by abstracting the Delta Lake protocol behind a clean API. 🛠️ Join this session to explore how @ClickHouseDB integrated 𝚍𝚎𝚕𝚝𝚊-𝚔𝚎𝚛𝚗𝚎𝚕-𝚛𝚜 into its single-binary C++ build system. 🚀 🎟️ Register: luma.com/OLAI-310 #openlakehouse #oss #deltalake #openlakehouseai #clickhouse

Q&A time - if you have any questions, please let us know!

Building a Scalable Usage Insights Platform with Delta Sharing x.com/i/broadcasts/1…

Hey all! Thanks for being here. Where are you tuning in from?