Michael Armbrust

@JorgeO electrifymyhome.com replaced my gas water heater with a heat pump. It's been pretty great!

Anyone have plumber recommendations for SF or the Bay Area? My gas water heater needed 3 repairs in the last month, another 3 in the 12mo prior, and it’s *still* having issues. I want to replace it. Also wondering if I should take the opportunity to electrify. Ty!

@jaceklaskowski I think we are just missing some context information as we start the extra threads we use to improve cluster utilization. We will fix it.

Found a way to dig deeper into #DeltaLiveTables 😎 And found the history of a published table all with nulls in userId and userName yet it's known who ran the pipeline?! 🤔 Should be easy to add it or I'm missing a bigger picture /cc @michaelarmbrust #Databricks

@jaceklaskowski @Falydoor Yeah, that is confusing. We should fix the icon. Longer term we are working on the ability for you to attach the notebook to your DLT pipeline cluster, so you can get interactive feedback on syntax or analysis errors.

@Falydoor One can consider DLTs as scheduled notebooks so LGTM.

Just came across this button "Delta Live Tables" in a #Databricks notebook Reminds me days with #IBM #WebSphere dev tools where writing code wasn't as cool as I always wanted 😏 (I'm not losing hope that there're some places for some code writing for DLTs, e.g. in #Python)

Insightful benchmark of Linux Foundation Delta Lake and Apache Iceberg by @BrooklynData that shows Delta is up to 8x faster in workloads with data updates. Most storage benchmarks only test reads, but with updates, care is needed to maintain performance. brooklyndata.co/blog/benchmark…

@psyfe @__apf__ Me too!

fifteen years ago my husband took me on a first date, one thing led to another and now he's building a large tree house

.@DeltaLakeOSS just got even better. Meet #DeltaLake 2.0, now *entirely* open source. @michaelarmbrust shares how the latest features improve performance & manageability. #DataAISummit

@jaceklaskowski @bigklata That is not exclusively true. Partitions can also help, for example with a SELECT DISTINCT <partition_column> and other aggregations, however, you are generally right that if the query is SELECT * you have to read the whole table definitionally.

@bigklata They do iff your queries use filters/WHEREs 😉

Kept hearing about data partitioning as a way to make many reads from a #DeltaLake table faster *somehow* but...that's what I "discovered" today...without WHERE clause (that #SparkSQL can push down) the whole table is scanned anyway (with or without partitions).

Databricks just set a new record on the official TPC-DS data warehousing benchmark, showing that a lakehouse system based on open data formats can outperform previous DW systems. Don't listen to folks who say open means bad performance! databricks.com/blog/2021/11/0…

As the #lakehouse gains momentum, we've been getting a lot of questions about what it means and how it compares to other architectures. Here are some answers to these and other common questions! databricks.com/blog/2021/08/3…

@__apf__ @hhhiryuu should probably try this tactic too. I like pepperoni :)

I'm seriously considering equipping people with deli meats and then "coincidentally" walking by them on the sidewalk

my pandemic pup is afraid when strangers get closer than 6ft, since that's been the rule his whole life 🤦🏻‍♀️ he warms up after a few min, but it isn't ideal. how do I fix him?

@technology @emilychangtv Why do you have to pick between diversity and merit? Make your work env. inclusive, decrease bias in hiring, as a leader don't make statements that alienate large groups. That'll give you a competitive advantage to a talent pool that won't join unwelcoming companies.

Check out @matei_zaharia and @alighodsi awesome talk to learn more youtube.com/watch?v=HQRusx…

Unlock your data and securely query it anywhere with #deltasharing on @DeltaLakeOSS!

And if you want to try it out, sign up for the preview! databricks.com/product/delta-…

If you want to jump into the details check out our docs: docs.databricks.com/data-engineeri…

Really excited to announce Delta Live Tables, a system that turns your SQL queries and pyspark DataFrame code into production-ready ETL pipelines! We've been working on this for almost 3 years, and it powers some of our largest pipelines internally. youtube.com/watch?v=fJhlTs…

@palvaro Yay! Congrats!

Did you know you can now use Delta Lake straight from #Python? Just a quick: pip install deltalake pandas

@bgawalt @dr_kuhalla Haha, I think its always allowed to define something as "not a weed" :)

@dr_kuhalla @bgawalt I had a similar question as clover is going crazy on my citrus tree. Apparently not all clover does nitrogen fixing though, TIL. gardensalive.com/product/ybyg-t…