Unity Catalog

With UC managed tables, teams unlock: 🔸 𝗨𝗻𝗶𝗳𝗶𝗲𝗱 𝗴𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲: Unity Catalog centralizes access control, replacing fragmented storage-level policies. This simplifies how teams ensure all engines access data in a governed, consistent manner. 🔸 𝗦𝘁𝗮𝗻𝗱𝗮𝗿𝗱𝗶𝘇𝗲𝗱 𝗱𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝘆: Unity Catalog provides stable logical table identifiers, eliminating the need for clients to depend on physical storage paths for discovery. 🔸 𝗘𝗳𝗳𝗼𝗿𝘁𝗹𝗲𝘀𝘀 𝘁𝗮𝗯𝗹𝗲 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻𝘀: By automating storage tuning and credential management, Unity Catalog removes the burden of manual operational maintenance from data teams. 🔸 𝗛𝗼𝗹𝗶𝘀𝘁𝗶𝗰 𝗮𝘂𝗱𝗶𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆: Metadata and permissions are centralized in a single interface, allowing for high-level oversight of ownership and access instead of parsing low-level storage logs. 🔸 𝗘𝗻𝗳𝗼𝗿𝗰𝗲𝗮𝗯𝗹𝗲 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁𝘀: Unity Catalog can authoritatively validate or reject schema and constraint changes, preventing incompatible updates that could compromise data integrity or break downstream workloads. 🔸 𝗙𝗮𝘀𝘁𝗲𝗿 𝗾𝘂𝗲𝗿𝘆 𝗽𝗹𝗮𝗻𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗳𝗮𝘀𝘁𝗲𝗿 𝘄𝗿𝗶𝘁𝗲𝘀: Unity Catalog delivers table metadata directly to Delta clients, bypassing cloud storage requests to significantly reduce metadata latency and accelerate query planning and writes.

Simplified Governance with Catalog-Managed Tables 🌐 Unity Catalog 0.4.0 introduces support for UC managed tables, enabling data teams to centrally govern, discover, access, and audit their data through Unity Catalog. Instead of relying on scattered storage paths, separate credentials, and manual maintenance, teams can rely on Unity Catalog as the single logical system of record for their data estate. Leverage UC managed tables to strengthen governance, improve performance, and build on the most modern open catalog for the data and AI era. 📖 Check out the announcement and implementation details on the Unity Catalog blog: lnkd.in/eZWkMBaR #unitycatalog #governance #opensource #catalogs #deltalake

We are excited to announce 𝗨𝗻𝗶𝘁𝘆 𝗖𝗮𝘁𝗮𝗹𝗼𝗴 𝟬.𝟰.𝟬 which includes exciting new features and many bug-fixes and improvements! 🎊 Check out some of the highlights 👇 𝗨𝗖 𝗦𝗲𝗿𝘃𝗲𝗿 ⚙️ Storage Credentials for AWS ⚙️ External Locations for AWS ⚙️ Managed storage location for catalogs and schemas 𝗨𝗖 𝗦𝗽𝗮𝗿𝗸 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗼𝗿 ⚡ Credential Renewal Enabled by Default ⚡ Support for Spark 4.1 and Delta 4.1 ⚡ Atomic CTAS for Delta Tables in UCSingleCatalog 𝗨𝗖 𝗔𝗜 🤖 DSPY Integration with AI Functions A huge thank you to the awesome community who made this release possible! 📖 Full release notes: github.com/unitycatalog/u… #unitycatalog #opensource #oss

We’re excited to announce the release of Unity Catalog v0.3.1! 🎉 This release includes exciting new features and many bug-fixes and improvements. Version 0.3.1 focuses on three major areas: 🔹 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝗱 𝗝𝗮𝘃𝗮 𝗰𝗹𝗶𝗲𝗻𝘁 𝗔𝗣𝗜 𝘄𝗶𝘁𝗵 𝗢𝗔𝘂𝘁𝗵 𝘀𝘂𝗽𝗽𝗼𝗿𝘁: designed for reliability and extensibility in production environments. 🔹 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰 𝗰𝗿𝗲𝗱𝗲𝗻𝘁𝗶𝗮𝗹 𝗿𝗲𝗻𝗲𝘄𝗮𝗹: to support long-running workloads cross cloud platforms. 🔹 𝗨𝗖-𝗺𝗮𝗻𝗮𝗴𝗲𝗱 𝗗𝗲𝗹𝘁𝗮 𝘁𝗮𝗯𝗹𝗲𝘀: this enables Unity Catalog to coordinate table storage and commits centrally. This release is the result of contributions from our growing open-source community. A big thank-you to everyone who reported issues, submitted pull requests, reviewed code, and shared feedback! 🔗 Dive into the release notes for the full list of highlights: github.com/unitycatalog/u… #unitycatalog #opensource #oss #catalog

Managing data pipelines at scale is complicated, often resulting in the data silo problem—where valuable assets are spread across systems. This makes it difficult to track, secure access, and scale cleanly. The solution is a clear structure paired with centralized governance. ✅ The Medallion Architecture structures your data into three distinct layers: 🥉 𝗕𝗿𝗼𝗻𝘇𝗲: Raw, ingested data. 🥈 𝗦𝗶𝗹𝘃𝗲𝗿: Cleaned, enriched data. 🥇 𝗚𝗼𝗹𝗱: Business-level data, ready for reporting. Pair this framework with Unity Catalog, and you get a unified system to manage, govern, and organize your entire data flow. 🔗 Walk through how this works: unitycatalog.io/blogs/building… #UnityCatalog #MedallionArchitecture #DataGovernance #OpenSource

⚡️ From vector DB to multimodal lakehouse at petabyte scale. Join us in Mountain View on Nov 13 for “Scaling Multimodal AI Lakehouse with Lance & LanceDB” with Chang She (@lancedb) at the Open Lakehouse + AI Mini Summit! Learn: 🔹 Low‑latency random access + search APIs (vectors, text, binaries) 🔹 Schema primitives across blobs + metadata for feature engineering 🔹 Hybrid search (vector + full‑text) for training/fine‑tuning at scale Two exciting tracks, one epic afternoon. 👏 #OpenLakehouse #AI #LanceDB #VectorDB #DataEngineering

Join us at Open Lakehouse + AI Paris on November 24, 6:30–10PM — co-located with the Forward Data Conference! 🇫🇷 Secure your spot now ⬇️ luma.com/OLM-1124 We’re bringing together data innovators and open source contributors for an evening packed with insight and inspiration. Hear talks from: ✅ Alexandre BERGERE (@DataGalaxy / Datalex) on Building a Scalable Usage Insights Platform with Delta Sharing ✅ Bartosz Konieczny (@waitingforcode) on Design Patterns for the Open Lakehouse ✅ Youssef Mrini & El Ghali Benchekroun (@Databricks) on The Future of Open Table Formats & @unitycatalog_io Food, drinks, networking — and plenty of new ideas (& swag) to take home. 🌟 #opensource #deltalake #oss #apacheiceberg #unitycatalog #lakehouse #openlakehouse #ai

🎤 @wslulciuc (oleander), hosted by Lisa N. Cao (@Databricks) #openlakehouse #datalineage #observability #oleander #AI #opensource

Missed it live❓See how an observability‑first Telemetry Lake correlates data movement with system behavior to detect issues, diagnose root causes, and adapt in real time—why correlation > coverage and why the first ~150 characters of signal matter. Watch: youtube.com/watch?v=r04iCv… What’s inside: @OpenLineage + @opentelemetry + an LLM reasoning layer that turns noisy lineage and traces into prioritized actions to cut TTD/TTR and reduce blast radius. #openlakehouse #datalineage #observability #AI

Where does the feature platform fit in a world increasingly dominated by AI? At Open Lakehouse + AI Mini Summit, Hao Xu (Apple) shares how Feast is evolving beyond a feature store into a full feature platform for AI—bridging data, models, and applications through innovations like Compute Engine, Feast for RAG, and On-Demand Feature Views. Don’t miss this deep dive into how foundational feature architecture continues to drive real-world AI innovation. 📍 Mountain View, CA 🗓️ Nov 13 🕦 12:00 - 4:30PM PT 🔗 Secure your spot: luma.com/OLMS-1113 #opensource #oss #unitycatalog #openlakehouse #ai

🚨 Exciting news: @starburstdata has announced GA support for Unity Catalog, enabling users to read and write to any UC managed table — @DeltaLakeOSS or @ApacheIceberg — using industry-standard open APIs! 🚀 What you should know: ✅ 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗲𝗱 𝗴𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 𝗲𝘃𝗲𝗿𝘆𝘄𝗵𝗲𝗿𝗲 — UC permissions are enforced wherever you query, with Starburst authenticating via OAuth 2.0 for per-user, secure access. ✅ 𝗨𝗻𝗶𝗳𝗶𝗲𝗱 𝗰𝗼𝗺𝗺𝗶𝘁 𝗰𝗼𝗼𝗿𝗱𝗶𝗻𝗮𝘁𝗶𝗼𝗻 — UC acts as the single commit coordinator for Delta Lake catalog-managed writes, enabling consistency and multi-table transactions across engines. ✅ 𝗢𝗻𝗲 𝗼𝗽𝗲𝗻, 𝗴𝗼𝘃𝗲𝗿𝗻𝗲𝗱 𝗹𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 — A single catalog for multi-engine reads/writes, advanced governance, and open table formats — no silos, no lock-in. The open lakehouse continues to grow through community-driven innovation. With Starburst joining engines and tools like @duckdb, @ClickHouseDB, @anyscalecompute, @langchain, and @daftengine, developers can now operate with consistent governance, broad interoperability, and transparent metadata standards across the entire data and AI lifecycle. 🔗 Learn more: starburst.io/blog/starburst… #opensource #oss #unitycatalog #starburst

The open source and data community is coming together in Mountain View, CA! 💥 Join us Nov 13 (12–4:30PM PT) for the 𝗢𝗽𝗲𝗻 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 + 𝗔𝗜 𝗠𝗶𝗻𝗶 𝗦𝘂𝗺𝗺𝗶𝘁, featuring two tracks packed with insights on AI infrastructure, context engineering, and the future of interoperable data systems. Lunch, swag, and great conversations included! Stick around for the @ApacheSpark Happy Hour (5–6:30PM PT) — the perfect way to wrap up a day of learning and community. 😎 🎟️ RSVP here: luma.com/OLMS-1113 #opensource #oss #unitycatalog #apachespark #deltalake #apacheiceberg #ai

📣 Get ready for the next Open Lakehouse + AI webinar! Join @wslulciuc, Co-Founder & CEO of @OleanderHQ, hosted by @lisancao from @databricks, will explore why the future of data lineage isn’t another graph—it’s a reasoning layer that connects lineage, telemetry, and AI. Learn how Oleander’s Telemetry Lake unifies @OpenLineage, @opentelemetry, and #LLM reasoning to help data teams detect, diagnose, and adapt in real time—building the always-on-call data engineer. 🚀 🗓️ October 30 🕝 9:00AM PT 🔗 Register: luma.com/openlakehouse-… #opensource #oss #openlakehouse #ai #oleander

Open Lakehouse + AI spotlight: Hannes Mühlheisen shares how anyone can contribute to @duckdb and DuckLake—both MIT-licensed, open source, and welcoming PRs and community engagement on GitHub. Check out this clip to hear how easy it is to get involved and start contributing today. 🚀 What’s next?​ 🔹 October 30: Webinar — The Failed Promises of Data Lineage: Why More Metadata Isn’t the Answer​ 🔹 November 13: Mini Summit — Mountain View​ 🔹 November 24: Meetup — Paris​ 🔗 RSVP and details: luma.com/openlakehouse #opensource #oss #openlakehouse #ai #ducklake #duckdb

Open Lakehouse + AI brings builders and practitioners together to share how open lakehouse and AI meet in the real world—through adoption stories, hands‑on patterns, and collaborative learning. 🌎 Join a global community shaping the future of data and AI! What’s next? 🔹 𝗢𝗰𝘁𝗼𝗯𝗲𝗿 𝟯𝟬: Webinar — The Failed Promises of Data Lineage: Why More Metadata Isn’t the Answer 🔹 𝗡𝗼𝘃𝗲𝗺𝗯𝗲𝗿 𝟭𝟯: Mini Summit — Mountain View 🔹 𝗡𝗼𝘃𝗲𝗺𝗯𝗲𝗿 𝟮𝟰: Meetup — Paris 🔗 RSVP and details: luma.com/openlakehouse A look back at our recent Open Lakehouse + AI Amsterdam meetup—huge thanks to the speakers, volunteers, and everyone who joined us! 👇📸 #openlakehouse #ai #lakehouse #unitycatalog #apacheiceberg #deltalake #apachespark #oss #openlakehouseai

#UnityCatalog helps avoid data silos by offering a single unified system to manage, govern, and organize all your data and AI assets—without locking you into a specific table format or query engine. 🚀 Define access rules once, keep metadata clean and centralized, and easily trace how data flows through your organization. Teams can continue to use their preferred tools and table formats, without compromising on efficiency or security. 🔗 Learn more: unitycatalog.io/blogs/avoid-da… #opensource #oss #unitycatalog #silos

What happens when robust, deterministic lakehouse design meets the power of AI agents? 👉 Reduced tool fragmentation 👉 Safer, reproducible automation 👉 Data teams enabled for true innovation On Tuesday, October 7 at 9AM PT, @Bauplan_labs founders @GreCo_CiRo and @jacopotagliabue will break down how function-based execution and Git-for-Data semantics pave the way for autonomous agentic workflows—cutting complexity and multiplying impact. 🔗 Register here to reserve your spot: luma.com/OLAI-107 #openlakehouse #opensource #oss #aiagents

At Open Lakehouse + AI Amsterdam last month, @Sammy_Sidhu —co-creator of the @daftengine multi-modal query engine and CEO of Eventual—shared how his open source journey began. 🚀 Ready to shape the future of data, open source, and AI together? Join us at our next Open Lakehouse + AI events! 🔹 Oct 7: Webinar — From Functions to AI Agents: Reimagining the Lakehouse for an Agentic Future 🔹 Nov 13: Mini Summit | Mountain View, CA 🔹 Nov 24: Open Lakehouse + AI | Paris, France Secure your spot: register now on Luma! 🔗 luma.com/open-lakehouse #OpenLakehouseAI #OpenSource #Daft #DeltaLake #Lakehouse #OSS

🚀 Open Lakehouse + AI events are dedicated to advancing open lakehouse and AI through adoption, collaboration, and sharing real-world use cases. Explore what’s coming up to learn, connect, and collaborate with a global community passionate about shaping the future of data and AI. 🔹 𝗢𝗰𝘁𝗼𝗯𝗲𝗿 𝟳: Webinar | From Functions to AI Agents: Reimagining the Lakehouse for an Agentic Future 🔹 𝗡𝗼𝘃𝗲𝗺𝗯𝗲𝗿 𝟭𝟯: Mini Summit | Mountain View 🔹 𝗡𝗼𝘃𝗲𝗺𝗯𝗲𝗿 𝟮𝟰: Meetup | Paris 🔗 RSVP & learn more at luma.com/open-lakehouse #opensource #lakehouse #openlakehouse #ai #deltalake #oss

On Tuesday, October 7 at 9AM PT, join @GreCo_CiRo and @jacopotagliabue of @Bauplan_labs for the next Open Lakehouse + AI webinar: “𝗙𝗿𝗼𝗺 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀 𝘁𝗼 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁𝘀: 𝗥𝗲𝗶𝗺𝗮𝗴𝗶𝗻𝗶𝗻𝗴 𝘁𝗵𝗲 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 𝗳𝗼𝗿 𝗮𝗻 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗙𝘂𝘁𝘂𝗿𝗲!” Why attend? 🔹 Learn how AI agents can manage ingestion, testing, and deployment. 🔹 Discover how function-based execution and Git-for-Data semantics make lakehouses agent-ready. 🔹 Explore how to unify analytics, ML, and governance while removing tool complexity. 📅 Don’t miss it—save your spot today: luma.com/OLAI-107 #openlakehouse #opensource #oss #aiagents