SDF Labs

We’re thrilled to announce that SDF is joining forces with @dbt_labs! Together, we’re setting out to transform data development and analytics for the better. Read more about this exciting milestone here: getdbt.com/blog/dbt-labs-…

Join us at Seattle's first ever Datafusion meetup, sponsored by SDF Labs and Spice AI. Monday, October 14th, 5:30p to 8:30p lu.ma/tnwl866b

Pretty excited about this one. @SDFLabs + @dagster is a huge step forward

#Dagster brings you fine-grained orchestration and deep observability. #SDF brings you advanced SQL transformations and static analysis. Together, you get the most efficient and transparent #data pipelines. Explore one of the best combos since PB&J here: bit.ly/4cv9ewm

🚀 Big news: @SDFLabs & @dagster Integration is here! 👉 Shorter feedback loops, reduced cloud costs, and true column-level lineage — all in one integrated toolchain. Ready to elevate your data game? Check out our latest blog post on the integration. blog.sdf.com/p/sdf-and-dags…

@davidgasquez @coltonpadden @dagster What would be your use case for native DuckDB support?

@coltonpadden @SDFLabs @dagster Cool! Been wanting to try it after seeing it used in the Dagster Open Platform. The main blocker is DuckDB support.... Does SDF plans on supporting DuckDB?

The @SDFLabs integration with @dagster as an alternative to dbt is an exciting development! Can't wait to see how this evolves.

@davidgasquez @coltonpadden @dagster Hey @davidgasquez! SDF has built-in execution, meaning we can execute SQL on your laptop at DuckDB-like speeds. You can think of it like DuckDB and DBT are baked into the same engine. You can try it in action with our tutorial: docs.sdf.com/tutorials/tuto…

This is a big day for @dagster and SDF! Congratulations to the folks at @dagster for a phenomenal 1.8 release. More on our integration coming next week.

If you aren’t using @SDFLabs to build your pipelines then you’re not doing it right

Returning to the office from holiday season, a time when most employees hope to pick up where they left off without issues, Patreon’s team discovered a mismatch between their realized customer signups and reported statistics from their data warehouse. When multiple data sets were found to not be updating properly, data engineers at Patreon quickly turned to @SDFLabs to provide visualized data lineage of their queries and models to locate the root cause. Learn how an industry leader simplifies their path to detailed insights within seconds by utilizing SDF Cloud: sdf.com/success-storie…

So excited to be working with this team!

Congrats to @SDFLabs on raising a $9m seed round and launching the public beta! For those unfamiliar, SDF was the dbt of Facebook. They make SQL more like a programming language by building the equivalent of Typescript to Javascript for SQL.

We are thrilled to introduce our latest feature to the cloud: Report Visualizations! 🎉 Ever wanted to quickly understand all unused columns in your warehouse? Or how about a breakdown of the queries with the most dependencies? Try SDF Reports! SDF Reports are SQL queries that run against SDF's information schema, enabling you to analyze your warehouse and metadata. They are an essential tool for tracking and analyzing business logic, datatypes, and metadata throughout your data warehouse. Our sleek new dashboard shows you and your entire team all your created reports at a glance, with the query output and key details all neatly displayed. Whether you’re running data deletion reports or performing dead column analysis, we’ve got you covered. We’ve designed this feature to empower members across all teams to unlock deeper insights into your data with a sleek, user-friendly interface. Learn more in our Cloud Reporting doc. docs.sdf.com/cloud/reporting

@ghalib Thanks! We’re excited for the future of data development and tooling, especially executable semantics. Join our community slack to keep up to date on all the releases and new features!

I like what @SDFLabs are doing. Data tooling is, desperately, an obvious fit for people who understand programming languages. But data people (almost by definition) rarely understand software engineering, let alone programming languages (see current tooling). I am excited.

Linqto’s Director of Data Engineering, Chris Hronek, experienced massive growth of his data warehouse, his team, and the complexity of requests he received -- a common occurrence for cutting edge data leaders. Chris found himself at a crossroads: pursue the conventional data stack to address the increasing demands on financial reporting analytics or find an alternative that met his data development goals. That’s when Chris turned to SDF to gain a complete understanding of his data warehouse and improve Linqto’s transformation layer. Learn how Chris and the Linqto team increased confidence in their decisions and the quality of their data warehouse. sdf.com/success-storie…

Looking for an easy way to learn about the next generation of data engineering tooling? SDF has a new tutorial with no setup required. Download the our command line interface (CLI) and walk through a series of short tutorials to explore SDF locally on your machine. Our tutorial series "Mom's Flower Shop" takes you through SDF's standout features with a dataset inspired by an actual flower shop run by SDF cofounder Elias DeFaria's mother. Tag a data engineer you'd trust with your flower shop data! 🌷 docs.sdf.com/tutorials/tuto…

You may have already seen it this previous week, but as a reminder SDF Labs announced a public beta of the SDF CLI! 🎉 docs.sdf.com/introduction/i… We have been moving at such a fast pace, releasing new features, patches, and bug fixes that we often near the end of the week and don’t take a moment to look back at the immense progress. We’re starting a semi-weekly series to solve that problem and benefit the users of SDF with a concise update on new docs, features, patches, upcoming events, and news from our team. To subscribe, join our new Slack Community (join.slack.com/t/sdfcommunity…) and our Email List (sdf.com). — Notes from this past week — Better support for local sources, performance improvements, and table & column descriptions propagate to Snowflake 10x faster schema download from Snowflake. This was a major bottleneck for several large scale customers pulling down metadata. Table and column descriptions in .sdf.yml files are now propagated up to Snowflake via the materialized query. This change was accomplished via the open source materializations lib. Workspace credentials are now live in the SDF Cloud

Thanks @headinthebox! We're so proud of the work we've put into SDF. It really is a culmination of our team's work at Meta on multi-dialect SQL and automated governance, contributions at Microsoft focused on compilers, runtimes, and scalable build systems, and academic studies in program correctness.

Love love love to see a startup getting funded that mentions "formal semantics", "static analysis", "compositional", "type system", "verifiable", "specification", "formalize", ... in their general availability announcement. Of course none of that comes as a surpise when you look at the founders' background and know their track record in making formal methods practical (i.e. dblp.org/pid/s/WolframS…). And 🙏 to the VC firms that took the bet to support this amazing work.