Manu Cornet
1.2K posts
@lmanul
Cartoonist; writer; feminist. Occasional attempts at being funny. Equally kind to CEOs and janitors. Blue tick because formerly @Twitter. 🐘 @[email protected]
This has been a year full of engineering excellence that sometimes can go unnoticed. Besides all the visible changes you see on our app, here are some of the most important improvements we have made under the hood. - Consolidated the tech stacks for For you, Following, Search, Profiles, Lists, Communities and Explore around a singular product framework. - Completely rebuilt the For you serving and ranking systems from the ground up, resulting in a decrease 90% reduction in lines of code from 700K to 70K, a 50% decrease in our compute footprint, and an 80% increase in the throughput of posts scored per request. - Unified the For you and video personalization and ranking models, which significantly improved video recommendation quality. - Refactored the API middleware layer of our tech stack and in doing so simplified the architecture by removing more than 100K lines of code and thousands of unused internal endpoints and eliminating unadopted client services. - Reduced post metadata sourcing latency by 50%, and global API timeout errors by 90%. - Blocked bots and content scrapers at a rate +37% greater than 2022. On average, we prevent more than 1M bots signup attacks each day and we’ve reduced DM spam by 95%. - Shutdown the Sacramento data center and re-provisioned the 5,200 racks and 148,000 servers, which generated more than $100M in annual savings. In total, we freed up 48 MW of capacity and tore down 60k lbs. of network ladder rack before re-provisioning it to other data centers. - Optimized our usage of cloud service providers and began doing much more on-prem. This shift has reduced our monthly cloud costs by 60%. Among the changes we made was a shift of all media/blob artifacts out of the cloud, which reduced our overall cloud data storage size by 60%, and separately, we succeeded in reducing cloud data processing costs by 75%. - Built on-prem GPU Supercompute clusters and designed, developed, and delivered 43.2Tbps of new network fabric architecture to support the clusters. - Scaled network backbone capacity and redundancy, which resulted in $13.9M/year in savings. - Started automated peak traffic failover tests to validate the scalability and availability of the entire platform continuously.
