Laurent Paris retweetledi
Read how @Microsoft makes data lineage extraction possible for #AzureDatabricks and #MicrosoftPurview users
openlineage.io/blog/openlinea…
#Azure #DataLineage #DataAnalytics #DataGovernance
English
Laurent Paris
154 posts
Laurent Paris retweetledi
If you have @SnowflakeDB Enterprise and you like lineage, I have good news for you. You can now easily send it to your favorite @MarquezProject or any other @OpenLineage compatible consumer. (Hint: check my bio)
OpenLineage@OpenLineage
The OpenLineage Adapter offers Snowflake's enterprise users a powerful tool for analyzing their pipelines. Read our latest blog post to learn more: openlineage.io/blog/openlinea…
English
Today starts the next stage of our evolution! I'm looking forward combining the best of data orchestration and data observability at @astronomerio, and helping grow the @ApacheAirflow and @OpenLineage communities.
Astronomer@astronomerio
Astronomer has acquired Datakin, the data lineage tool from founders of the @OpenLineage and @MarquezProject. The move accelerates our shared goal of helping organizations build and manage reliable ecosystems that deliver trusted data. #blog hubs.ly/Q016zFzH0
English
Laurent Paris retweetledi
We interrupt our programming for an important message of Ross from @DatakinHQ vimeo.com/601590009
English
Data lineage captured in real time as it evolves is the key to true data observability.
Datakin@DatakinHQ
There are a lot of different ways to collect #datalineage metadata. We choose to observe it in real-time - here's why. datak.in/2WUix7X
English
This is a big milestone in the journey to bring true observability to the world of data. Thanks to all community members who have made this possible! #dataobservability #dataquality
Datakin@DatakinHQ
Congrats to @OpenLineage, which has just become an @LFAIDataFdn sandbox project! Consistent collection of lineage metadata is hard, and requires a broad collaborative effort.
English
Laurent Paris retweetledi
If you don’t know what you need to know when your pipeline is on fire, you are likely to get burned. See the difference #datalineage can make the next time you're on-call.
datak.in/3rgH9TN
English
Laurent Paris retweetledi
We're excited to announce that the Marquez integrations for Airflow, Spark and DBT will be the official integrations for @OpenLineage! 🎉 To learn more about the high-level timeline of when the integration will be moved over, see: github.com/OpenLineage/Op…
English
Laurent Paris retweetledi
Once we have data lineage, we can start to work with platforms that provide an operational degree of introspection into data logs, metrics and traces i.e. all the elements of data mechanics that give us a window into that thing we seek most in cloud - observability.
Me, 2021
English
Laurent Paris retweetledi
@data_mesh_learn @montecarlodata No. A rigorous record is required, not notes. Provenance requirements. And we do document it. In code. And there is an execution plan with all of this information. All the information is in Spark. We just need to save it. Sounds like @OpenLineage and Marquez do exactly this.
English
Laurent Paris retweetledi
There are a lot of similarities between the goals of @OpenTelemetry and @OpenLineage - this post (on the shiny new OpenLineage site!) explores them in detail
Julien Le Dem@J_
I blogged on the parallels and differences between observability for Data Pipelines and Observability for Services. "How @OpenLineage takes inspiration from @OpenTelemetry" openlineage.io/blog/openlinea…
English
Laurent Paris retweetledi
Great post from @HightouchData about @OpenLineage
paving the way to the future of data observability.
hightouch.io/blog/exploring…
English
Laurent Paris retweetledi
Data without context is just technical debt. That’s why lineage matters - if you haven’t spent much time thinking about how to track #datalineage, this podcast about @openlineage is a great place to start.
Datakin@DatakinHQ
Episode 187 of the @DataEngPodcast is live, featuring our CTO @J_ providing a great overview of #datalineage with @OpenLineage dataengineeringpodcast.com/openlineage-da…
English
Remembering the time at Yahoo! I worked on a project with @parisl called Global Local. No joke.
English
@mattturck All symptoms of the Great Dispersion profgalloway.com/the-great-disp…
English
The magnitude of the assault on Wall Street is kinda crazy when you put it all together:
Hedge funds —-> Robinhood
Bloomberg —> Reddit
Mutual funds, gold etc —> Crypto
IPO bankers —> SPACs
Early innings, but giant dislocation ahead
English
Laurent Paris retweetledi
We are excited to launch @OpenLineage and work cross functionally with contributors of open source projects in the data ecosystem to enable the development of tools and solutions in terms of data operations, governance, and compliance.#DataOps #DataLineage
OpenLineage@OpenLineage
Today is the official launch of @OpenLineage, a new open source project to define a flexible industry standard for data lineage. Thank you for the collaboration @drewbanin @mistercrunch @AbeGong @MandyChessell @wesmckinn @zhamakd datakin.com/2020/12/18/int…
English
Laurent Paris retweetledi
Are you looking for a "hello world" approach for #datagovernance?
Take a look at my latest article on available metadata solutions.
p.s. @MarquezProject is a great place to start!
dataplatformschool.com/blog/w0y8g0-th…
English
Can't agree more on the need to be more warm-hearted these days
Dalai Lama@DalaiLama
You are human beings, I'm a human being; you want a happy life, and I want a happy life. But I think that aim will not be fulfilled only on the basis of material values; what we all want is affection and what we all need is to be more warm-hearted.
English