Apache Wayang

Apache Wayang isn’t just limited to FedML and Data Federation; it also enhances our taste and lifestyle. Solving real world data science problems! youtube.com/watch?v=4f24QS…

@mapredit Thank you, and all other committers and contributors to make this happen. Over four years research, testing, coding. FedAI is coming! Open source always wins.

@apachewayang 1.0.0 build successful! The backbone for AI, decentralized data processing to access data in silos a dev can't reach typically.

@apachewayang v. 1.0.0 successfully compiled. The backbone for ASI / AGI infrastructure, devs can now reach data which is typically not in reach, like protected silos, and execute processing at the source, in-situ. FedAI is coming soon!

LLM's struggle to show their business benefit. Why? Data access. AI needs the right data, in a suitable format at the right time. Read how @apachewayang helps to solve that: wayang.apache.org/blog/wayang-fe…

Our newest blogpost, written by Wayang developer Juri Petersen, explains how python works with Apache Wayang, and how we enable JSON conform UDF within a wayangPlan. #data #opensource #apachewayang wayang.apache.org/blog/wayang-py…

Read part 1 of the blog series, written by Dr. Mirko Kämpf from ecolytiq, about implementing a cross-organization decentralized data processing scenario. #apachewayang #kafka #data wayang.apache.org/blog/kafka-mee…

EarthObservation / ESA / Copernicus: AGORA plugin (Apache Wayang): meetingorganizer.copernicus.org/EGU23/EGU23-17…

Integrate AI with IoT, the federated way: youtube.com/watch?v=dOnsEf… #iot #spark #bigdata

We updated our website -> wayang.apache.org <-. More documentation, much more easier to contribute and add PRs.

Apache Wayang now enables AI/ML training and inference in Spark for federated learning at the enterprise level. The team just merged PR388: github.com/apache/incubat…

@DatabloomAI Thank you!

We prepare a code push to @apachewayang to enable multiple spark and jdbc data queries in one plan. This is federated learning on industrial level, across databases and data lakes. Model training at a fraction of the current costs coming soon! #opensource #platform #data

We merged 380: #380 Added a new plugin to support different jdbc platforms and multiple instances of same DBMS. Wayang now adds support for multiple spark and jdbc context in one wayangPlan(). github.com/apache/incubat…

ML4ALL implementation merged, it's the code to our reseach paper, now merged and fully operable. github.com/apache/incubat…

Now you can run kmeans natively on sparkML! This improvement speeds ML operations massively up. Additionally, with wayang you can have multiple _different_ spark version in one plan, that makes wayang really powerful for cross-spark ML. github.com/apache/incubat…

Apache Wayang: A Unified Data Analytics Framework - research paper now released with Sigmod Record => sigmodrecord.org/publications/s…

Wayang AgoraEO Plugin: The Framework for Scalable EO Workflows "We recently proposed AgoraEO, a decentralized, open, and unified ecosystem, where users can find EO elements, compose cross-platform EO pipelines, and execute them efficiently. ui.adsabs.harvard.edu/abs/2023EGUGA.…

#CommunityOverCode NA, our flagship event, is only TWO MONTHS away! Join us in Halifax 🇨🇦 for four days of sessions, with tracks focusing on Search, Big Data, Streaming, IoT, Community, Fintech, and more. 🎟️ Register now! bit.ly/3D9fKK7

#DynamicPartitioning in Apache Doris It means to automatically create and delete data partitions based on the user-specified rules. You can enable it upon data creation and specify the partition TTL (time to live) in the statement. #database #DataScience #Analytics #SQL

Apache Wayang enables cross platform data processing, star it via: github.com/apache/incubat… #dataprocessing #bigdata #analytics #hybridcloud #developers via @apachewayang