Sijie

1.5K posts

Sijie banner
Sijie

Sijie

@sijieg

Co-founder & CEO of @streamnativeio | @apache_pulsar, @asfbookkeeper. In a previous life: @streamlio @twitter @yahoo

Bay Area Katılım Nisan 2008
450 Takip Edilen1.3K Takipçiler
Sijie
Sijie@sijieg·
Just listened to Jay’s keynote at Current Summit London. Unified batch and streaming? That’s old news. Snapshot queries? That’s just querying Iceberg tables… come on. You can already do that with #Ursa by querying Ursa topics’ lakehouse tables using #Flink, #Spark, or any engine you like—no need to wait for Confluent’s snapshot query to GA. Want to see it in action? Stop by the StreamNative booth. We’ll show you how Ursa + open source Flink gives you fast, flexible snapshot queries—today. #Current25
English
0
0
1
208
Sijie
Sijie@sijieg·
I just kicked off a new blog series on AI agents! First post tackles a pain point: chatbots powered by stale data and sit in silos. 🤖➡️🌊 How do we feed them real‑time streams and get them talking at scale? Check it out & tell me what you think 👉 streamnative.io/blog/ai-agents…
English
0
1
4
208
Sijie
Sijie@sijieg·
Very excited to announce that @streamnativeio now integrates with @databricks Unity Catalog—the first & only data streaming engine to natively support it! Seamlessly ingest streaming data into Delta Lake & Iceberg for real-time analytics with unified governance by Unity Catalog. streamnative.io/blog/seamless-… #DataStreaming #UnityCatalog #Lakehouse #AI #RealTimeAnalytics #ApacheKafka #DeltaLake #ApacheIceberg
StreamNative@streamnativeio

StreamNative's Ursa Engine now integrates with @databricks Unity Catalog, bringing real-time streaming to the Lakehouse! ✅ Stream to Iceberg & Delta Lake ✅ Better data governance & AI workflows ✅ Faster, cost-effective real-time analytics hubs.ly/Q0351NYT0 #RealTimeData #Databricks #Lakehouse #AI #Streaming

English
0
0
2
403
Sijie retweetledi
StreamNative
StreamNative@streamnativeio·
🚀 Excited to announce our Technology Select Tier partnership with @SnowflakeDB Together, we’re simplifying real-time analytics & AI: ✔️ Faster performance ✔️ Lower cloud costs ✔️ Scalable, streamlined architectures Learn more: hubs.ly/Q02_rXR60 #DataStreaming #AI #Snowflake
StreamNative tweet media
English
0
1
1
208
Sijie
Sijie@sijieg·
While #BYOC has become increasingly popular, we've seen debates about which approach to BYOC is best. In our view, whether it's BYOC, Dedicated/Serverless, or self-managed, what organizations need is a Portable Data Plane. Checkout our blog post: streamnative.io/blog/byoc2-por…
Sijie tweet media
English
0
0
2
240
Sijie
Sijie@sijieg·
While "Shift-Left" is a hot topic in data strategies today, particularly for advancing real-time data streaming architectures, it's important to look at it within a broader context. We believe in a more holistic approach. In today’s data-driven world, Shift-Left alone isn’t enough. For a truly comprehensive strategy, you need to balance Shift-Left with other architectures like Lakehouses. Data streaming is critical for real-time generative AI, but the real foundation for these applications lies in the combination of data streaming and lakehouses. This synergy is what we call the Streaming-Augmented Lakehouse (SAL)—akin to how Retrieval-Augmented Generation (RAG) enhances large language models (LLMs). Just as RAG augments AI, SAL enhances lakehouses with real-time streaming capabilities, creating a dynamic data ecosystem for real-time AI. But let’s be clear: SAL is not just another Lambda Architecture. The essence of SAL is in its headless, multi-modality data storage that allows you to store one copy of data and present it as a stream or table depending on your use case. It moves ingestion and computation left, achieving low latency while ensuring data quality and governance. With SAL, businesses get the flexibility of multiple modalities (stream or table), protocols (Kafka or Pulsar), and semantics (competing queues vs. sequential streams) to meet diverse business needs. In the AI age, the future of data strategy demands more. Businesses need a new data foundation that combines the real-time power of streaming with lakehouses. SAL is that foundation. It enables real-time data to continuously fuel AI systems while also managing the vast historical datasets required for training and improving models. By adopting SAL, enterprises can bring real-time insights directly into their lakehouse architectures—acting on data as it arrives while maintaining governance, scalability, and deep analytics. We believe SAL is the key to unlocking the full potential of real-time Gen AI. Curious to learn more? Check out our latest blog post! 👉 streamnative.io/blog/introduci… And if you’re passionate about the latest trends in data streaming and AI, join us at the Data Streaming Summit on October 28-29 at the Grand Hyatt SFO. DM me if you’d like discounted tickets! eventbrite.com/e/data-streami… #ApacheKafka #ApachePulsar #Lakehouses #ShiftLeft #StreamingAugmentedLakehouse #GenAI
English
0
0
1
261
Sijie
Sijie@sijieg·
6 years ago, I never imagined #ApachePulsar would grow into what it is today. From an idea to solve real-time data challenges to a thriving community project @apache_pulsar it’s been an incredible journey. 🙌 Grateful for everyone who's been a part of this! 🚀 👉 streamnative.io/blog/celebrati…
English
0
2
9
428
Sijie
Sijie@sijieg·
Sad to hear that Upstash shut down its serverless Kafka service. If you're looking for an alternative, check out Serverless Kafka on @streamnativeio Cloud! Start with $200 in free credits—no credit card needed. Need more? DM me! Learn more: streamnative.io/blog/introduci…
Upstash@upstash

Hey friends, this tweet is an announcement to our Upstash Kafka users. When we founded Upstash, we identified messaging and task scheduling as the next problems to solve after state management. Initially, we considered Kafka the ideal solution due to its power as a messaging platform. This assumption seems correct as we've received positive feedback for our Kafka offering, and it's financially viable. However, we noticed significant friction for serverless developers using Kafka. Traditional message broker like Kafka is not well-adapted for the serverless use cases. This is why we had to make the difficult decision to halt new Kafka feature development to focus our resources on QStash and Upstash Workflow. Existing users can continue using all platform features, including connectors and the schema registry as normal for the next six months. No immediate action is required because we recognize that a migration plan needs time. After the six month period, we plan to deprecate Upstash Kafka. On the other hand, QStash and, recently, Upstash Workflow naturally evolved into tools specifically for serverless needs. To our surprise, we see an increasing number of customers use QStash for complex business logic orchestration, effectively using it as an alternative to Kafka. This strategic shift allows us to focus on solutions specifically for serverless environments. By focusing our limited resources, we can invest more heavily into QStash and Upstash Workflow. Also, this decision allows us to improve our Redis and Vector offering to provide a more cohesive ecosystem of tools. We want to be fully transparent about our intentions and want to make this transition as easy as possible for our Kafka users. If your Kafka use case can be covered with Upstash QStash, we will cover the entirety of your migration costs. If not, our professional support is always there to help with any of your questions. We appreciate your trust in us and are excited about the future we're building. We believe these changes will ultimately allow us to provide you with a better, more focused solution optimized for serverless. Read the full announcement here: upstash.com/blog/workflow-…

English
0
5
9
1.1K
Sijie
Sijie@sijieg·
Say hello to UniConn (Universal Connectivity)! After adding support for the Kafka protocol with Ursa engine, we've gone a step further by bringing in support for Kafka Connect. With UniConn, you can now run Kafka Connect connectors natively on StreamNative Cloud, using the same runtime and scheduler as Pulsar IO connectors. It’s all about making data streaming easier, no matter where your data lives! Check it out: streamnative.io/blog/revolutio…
Jay Kreps@jaykreps

15/ Trying to capture that ecosystem with fragmented, single vendor systems will fail. If you are a vendor, why would you build your connector for a single vendor platform like RedPanda Connect if you could instead target an open standard like Kafka Connect with massive adoption?

English
0
3
7
898
Sijie retweetledi
StreamNative
StreamNative@streamnativeio·
🎉 Exciting news! StreamNative is proud to announce Serverless, Universal Connectivity (UniConn) and a new Partner Program! 🚀 We're on a mission to democratize data streaming and make it accessible to organizations of all sizes. Learn more prnewswire.com/news-releases/…
English
0
1
2
245
Sijie retweetledi
StreamNative
StreamNative@streamnativeio·
Unifying the #Kafka and #Pulsar ecosystem with Universal Connectivity (UniConn). Our new solution that provides a consistent experience for connecting, processing, and monitoring data pipelines powered by both Kafka Connect and Pulsar IO. Learn more: hubs.ly/Q02PkhMX0
English
0
2
3
226
Sijie
Sijie@sijieg·
If you are in the SF Bay Area, don't miss the presentation from the Paypal team about their adoption journey with @apache_pulsar. Join us this Thursday evening in Sunnyvale! We have speakers from @PayPal @PingCAP @streamnativeio talking about all things data.
TiDB@TiDB_Developer

🌉For those of you in the SF Bay Area, we're joining forces with @streamnativeio for a meetup on June 20 in Sunnyvale. We’ll have speakers from @PayPal, StreamNative, and @PingCAP along with food and drinks. You can find more details and register for the meetup at meetup.com/real-time-data… 👈

English
0
3
10
637
Sijie
Sijie@sijieg·
Huge congrats to our friends at @tabulario and @databricks!!! I am really excited to see how these open table formats will converge. The open lakehouse format is a foundation for enabling data sharing across different teams, departments, and organizations. StreamNative’s #Ursa engine has integrations with Databricks / Delta Lake and Tabular / Iceberg as the lakehouse storage. We are excited to continue our collaboration with these two great teams (now one team) and integrate the best of both worlds in data streaming and data lakehouse.
Tabular (now part of Databricks)@tabulario

We are thrilled to announce that @databricks and Tabular are joining forces to solve lakehouse interoperability. We intend to work closely across the #ApacheIceberg and #DeltaLake communities to bring open table format compatibility to the #lakehouse. tabular.io/blog/tabular-i…

English
0
0
6
1.2K
Sijie
Sijie@sijieg·
We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities.
English
0
0
0
166
Sijie
Sijie@sijieg·
#Ursa is not a technology reimplementation for the current economy; it is the realization of our vision at StreamNative to facilitate seamless data sharing across teams, departments, and even organizations, irrespective of data origin. Along in the journey of developing Ursa, instead of developing one single compute engine to unify stream and batch processing, we democratize access to a shared data layer that supports multiple protocols & semantics, with a multi-modal storage layer, enabling diverse access methods within a multi-tenant framework.
English
1
0
0
200