Vanessa Vuibert

328 posts

Vanessa Vuibert banner
Vanessa Vuibert

Vanessa Vuibert

@V3_XD

#ApacheKafka tech lead at @Shopify. Views are my own 🇪🇨🇨🇦

Canada Katılım Şubat 2010
143 Takip Edilen211 Takipçiler
Vanessa Vuibert
Vanessa Vuibert@V3_XD·
@GergelyOrosz @ShopifyEng It's 66M messages per second. We use Kafka's monitoring: kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=([-.\w]+) #remote_jmx" target="_blank" rel="nofollow noopener">kafka.apache.org/documentation/…
English
1
4
90
48.7K
Gergely Orosz
Gergely Orosz@GergelyOrosz·
@ShopifyEng I assume this is 66M messages per minute, not per second. It would not add up with other load numbers (e.g. peak CDN load was 128M CDN requests per *minute*)
English
1
0
6
3.4K
Shopify Engineering
Shopify Engineering@ShopifyEng·
Powering global-scale commerce demands observability systems that can handle immense scale. During the BFCM peak, we processed an astounding 7TB of logs per second 🚀📊 Check out more BFCM highlights 🧵👇
English
9
31
206
39K
Vanessa Vuibert retweetledi
Shopify Engineering
Shopify Engineering@ShopifyEng·
Our Apache Kafka fleet smashed records this BFCM, handling over 66 million messages per second! That's 120% more streaming than 2023 🛍️📈
English
2
3
64
13K
Vanessa Vuibert
Vanessa Vuibert@V3_XD·
#apacheKafka at Shopify #BlackFriday peak growth through the years: - 2021 12.9M msg/sec - 2022 52% growth with 19.6M msg/sec - 2023 54% growth with 30.2M msg/sec - 2024 120% growth with 66.3M msg/sec
English
0
1
0
430
Vanessa Vuibert
Vanessa Vuibert@V3_XD·
A few more fun #apacheKafka peak stats during Black Friday for @ShopifyEng: - 120% growth from 2023 with 66.3 million messages per second - 4.89 PB of data - 6.3 ms latency
tobi lutke@tobi

Yesterday was an incredible day for our merchants and for Shopify: * Merchants drove a record-breaking $5 billion Black Friday in sales (GMV) * Peak sales / minute for yesterday = $4.6 million * Shopify's egress processed more than 173B requests * Peak requests per minute was at 284 million on edge, with more than 80 million on app servers pushing 12TB a minute * 45 million peak database queries per second, peak database writes was 7.6 million per second * We rely on Kafka. At peak we hit 66 million messages per second Let's keep going 🔥

English
1
1
4
1.4K
Vanessa Vuibert
Vanessa Vuibert@V3_XD·
A few more fun #apacheKafka average stats during Black Friday Cyber Monday at Shopify: - 1.5 trillion messages per day - 2.9 PB of data - 4.8 ms latency
Shopify Engineering@ShopifyEng

Our #ApacheKafka streaming infrastructure served 29 million messages per second at peak. That’s 45% growth from 2022 – and we did it more efficiently this year with 14% fewer brokers.

English
2
2
5
925
Vanessa Vuibert
Vanessa Vuibert@V3_XD·
I am once again a speaker this year at #Current23! I'll be talking about how to improve the resiliency of #ApacheKafka clusters with DNS traffic management and client wrappers. Thanks for having me back and I look forward to seeing you next week!
Vanessa Vuibert tweet media
English
0
0
0
174
Vanessa Vuibert retweetledi
Shopify Engineering
Shopify Engineering@ShopifyEng·
📈Our #ApacheKafka streaming infrastructure served 20 Million messages per second at peak, that’s a 52% growth from 2021!
English
1
6
85
0
Vanessa Vuibert retweetledi
Mehryar
Mehryar@MehryarM·
👋🏼 The streaming platform team @ShopifyEng had an absolute blast participating in #Current22 the past couple of days! With the event wrapped up, here are some of our talks that you should definitely check out when the recordings are available 👇🏼🧵:
English
3
6
17
0
Vanessa Vuibert retweetledi
poorly drawn lines
poorly drawn lines@PDLComics·
remember me
poorly drawn lines tweet media
English
7
371
3.1K
0
Vanessa Vuibert retweetledi
The Olympic Games
The Olympic Games@Olympics·
Neisi Dajomes of #ECU wins #Gold in the women’s weightlifting 76 kg! It’s the first time Ecuador have won two medals at the same Olympic Games and Dajomes becomes the first Ecuadorian woman to win an Olympic gold medal! @ IWFNet #StrongerTogether | #Tokyo2020 | #Weightlifting
English
99
2.5K
7.3K
0
Vanessa Vuibert retweetledi
Neil deGrasse Tyson
Neil deGrasse Tyson@neiltyson·
I wonder what Space Aliens would say, if told that Earthlings shift clocks by an hour in the summertime to fool themselves into thinking there's more sunlight in the day.
English
361
668
7.4K
0

Keşfet

@GergelyOrosz @ShopifyEng @chch_gye @yeh_ling @elonmusk @BarackObama @taylorswift13 @cristiano