Walaa Eldin Moustafa

@neelesh_salian @matsonj @matsonj, here are some slides about incremental maintenance with dbt slideshare.net/walaa_eldin_mo…. The code is linked from the slides.

@matsonj Check out Coral: My friend @w_moustafa leads the project. You both should sync if there is interest. twitter.com/neelesh_salian…

Talked to my CTO about potentially using dbt for some prod analytical tables and most of the conversation ended up being about “but why does it rebuild every dbt run”

@marsupialtail_2 @neelesh_salian @LinkedInEng @Captaintobs Coral has 2 equivalent IRs, one at the AST layer and another is at the logical layer (it does not deal with the physical layer). I think that makes it pretty flexible yet powerful and robust. More details in the slides @neelesh_salian shared.

@neelesh_salian @LinkedInEng @Captaintobs seems like they built a sqlglot with backend specific optimization

Coral: A SQL translation, analysis, & rewrite engine for modern data lakehouses I've seen this in action and they have a dbt plugin and a more efficient incremental materialization: github.com/linkedin/coral… I'll find the slides to share. engineering.linkedin.com/blog/2020/coral via @LinkedInEng

@joe_hellerstein @julianhyde There is basic support of recursion (no negation or aggregation). Implementing relational algebra front ends is straightforward (e.g., Pig Latin, SPARQL, Datalog). See relevant Datalog discussion here: mail-archives.apache.org/mod_mbox/calci…. Would love to explore linear algebra extensions.

Lazyweb (and @julianhyde): Most of the Apache Calcite docs including the SIGMOD paper focus on SQL and relational algebra. Is it being used to support other funky algebras and query languages, in the spirit of the Volcano/Cascades papers? If not, how extensible is it to that?