ApacheDataFusion

While developing @ApacheDataFusio, you can experiment with code changes and queries using the high-level datafusion-cli REPL, similar to chdb or duckdb CLIs. I find it much faster than running the whole test suite. Hint: enable Rust debug logs to peek into the internals.

Today's Database Building Blocks Seminar Speaker: @trent_hauck will present the Exon database system for bioinformatics workloads based on @ApacheDataFusio. Zoom talk open to public at 4:30pm ET. YouTube video available after: db.cs.cmu.edu/events/buildin…

incredible, incredibly dense talk on @ApacheDataFusio from @andrewlamb1111, ranging from context & overview to deep dive on the guts & insider trivia youtu.be/iJhRbDFJjbg?si…

Starting a "space time" week at @elastic, where we can work on whatever we want that is loosely related to our products. For me, this will be experimenting with @ApacheDataFusio to see how we can use/integrate it with Elasticsearch.

@IAmMattGreen @asc89 Also, on Sep 25th, @asc89 will be talking live about all that stuff on the next SF DataFusion meetup. Come and join us! lu.ma/05lyf19h

How does the term "DuckDB for streaming" make you feel? This week we are chatting with @IAmMattGreen and @asc89 founder of buff.ly/3XnAasn about embedded stream processing, Apache DataFusion, why fault-tolerance is hard and also why we might not need it as much as we think. Check the episode 👇 buff.ly/4gCdtd5

DataFusion python is getting much better and easier to use: datafusion.apache.org/blog/2024/08/2…

#InfluxDB Staff Engineer @AndrewLamb1111 will be at the @ApacheDataFusio Meetup in NYC on Sept. 17th! 🗽 We're looking forward to meeting the #DataFusion community, discussing the amazing work that's being done, and talking about the future of the project. 🔗 bit.ly/3WLY16f #InfluxDB

Join us for the first Seattle @ApacheDataFusio Meetup in Oct! And if you are interested in speaking submit your talk at forms.gle/WmJrqipu9JEXAB…

DataFusion Comet 0.2.0 released: datafusion.apache.org/blog/2024/08/2…

In this blog, we share our experience implementing StringView in the Rust implementation of #ApacheArrow and integrating it into @ApacheDataFusio, significantly accelerating string-intensive queries in the ClickBench benchmark by 20–200%. 🔗 bit.ly/3Z3UFwF #InfluxDB #Rustlang #DataFusion

New post! I talk with @philippemnoel about @paradedb, the experience of building as a PostgreSQL extension, pg_duckdb, pg_lakehouse, and more... materializedview.io/p/search-on-po…

Seminar Schedule: Sep 23: @ApacheDataFusio Sep 30: @andygrove_io→DataFusion Comet Oct 07: @ParadeDB Oct 21: @VoltronData→Theseus Oct 28: @WHERE_TRUE_TECH→Exon Nov 04: @synnadahq Nov 11: @InfluxDB Nov 18: @GlareDB Nov 25: @Greptime Dec 02: @DatabendLabs→OpenDAL

Huge @ApacheDataFusio v40 release, congrats! A nice call out for the @spice_ai contributed TableProviders repo too. datafusion.apache.org/blog/2024/07/2…

This is the architecture of Spice. Query engine: @ApacheDataFusio Materialization: @ApacheArrow @duckdb SQLite Storage/Catalog: Any data system inc. Parquet/Iceberg, Databricks+UC, Snowflake, PostgreSQL, FTP, GraphQL, Sharepoint AI/ML: Any model/AI backend inc. Local inference, OpenAI, Groq, Anthroptic, HuggingFace, etc. github.com/spiceai/spiceai

DataFusion 40.0.0 release announcement: datafusion.apache.org/blog/2024/07/2…

🎉 Thrilled to see @ApacheDataFusio included in @CRN’s 10 Coolest #OpenSource Software Tools of 2024! Congrats to creator @andygrove_io! ➡️ bit.ly/4cBuktL Read about #InfluxDB’s use of DF for the FDAP stack in this post from @AndrewLamb1111. ➡️ bit.ly/3LoaMgB

I can't imagine so many people come the first @ApacheDataFusio meetup in Hangzhou, people are interested in how to leverage DataFusion to speedup their system. This is my slides, hope it helps! slides.liujiacai.net/datafusion-in-…

We're hosting a meetup for @ApacheDataFusio on September 17 in NYC, w/ @andrewlamb1111 and @nerdsane. Office courtesy of @datadoghq Come hang and catch a glimpse of the future of data infra lu.ma/2iwba0xm

How Apple is using @OpenLineage and @MarquezProject to visualize lineage in @trinodb, @ApacheAirflow, @ApacheFlink and @ApacheSpark. youtube.com/watch?v=A7hj1M…