Gabriel Capparelli

How To Release A Mobile App

6 API Architecture Styles You Should Know. API architectural styles determine how applications communicate. The choice of an API architecture can have significant implications on the efficiency, flexibility, and robustness of an application. So it is very important to choose based on your application's requirements, not just what is often used. Let’s examine some prominent styles: REST A cornerstone in web services, REST leverages HTTP methods for streamlined operations and a consistent interface. Its stateless nature ensures scalability, while URI-based resource identification provides structure. REST's strength lies in its simplicity, enabling scalable and maintainable systems. Learn more about REST here: drp.li/what-is-a-rest… GraphQL Whilst REST uses multiple endpoints for each resource and necessitates multiple requests to obtain interconnected data; GraphQL uses a single endpoint, allowing users to specify exact data needs, and delivers the requested data in a single query. This approach reduces over-fetching, improving both performance and user experience. Learn more about GraphQL here: drp.li/graphql-how-do… SOAP Once dominant, SOAP remains vital in enterprises for its security and transactional robustness. It’s XML-based, versatile across various transport protocols, and includes WS-Security for comprehensive message security. Learn more about SOAP here: drp.li/soap-how-does-… gRPC gRPC is efficient in distributed systems, offering bidirectional streaming and multiplexing. Its use of Protocol Buffers ensures efficient serialization and is suitable for a variety of programming languages and use cases across different domains. Learn more about gRPC here: drp.li/what-is-grpc-z… WebSockets For applications demanding real-time communication, WebSockets provide a full-duplex communication channel over a single, long-lived connection. It's popular for applications requiring low latency and continuous data exchange. Learn more about WebSockets here: drp.li/webhooks-and-w… MQTT MQTT is a lightweight messaging protocol optimized for high-latency or unreliable networks. Its pub/sub model ensures efficient data dissemination among a vast array of devices, making it a go-to choice for IoT applications. Learn more about MQTT here: drp.li/automation-wit… API architectural styles are more than just communication protocols; they are strategic choices that influence the very fabric of application interactions. There is no best architectural style. Each offers unique benefits, shaping the functionality and interaction of applications. It's about making the right choice(s) based on your application's requirements. If you want to learn more about API development, the 2024 State of the API Report has just been released. Check it out for key trends and insights: drp.li/state-of-the-a… ~~ 📩 If you like this post, then you'll love our newsletter where we help you level up your engineering and system design skills. Subscribe here: blog.levelupcoding.com

Explaining 8 Popular Network Protocols in 1 Diagram. Network protocols are standard methods of transferring data between two computers in a network. 1. HTTP (HyperText Transfer Protocol) HTTP is a protocol for fetching resources such as HTML documents. It is the foundation of any data exchange on the Web and it is a client-server protocol. 2. HTTP/3 HTTP/3 is the next major revision of the HTTP. It runs on QUIC, a new transport protocol designed for mobile-heavy internet usage. It relies on UDP instead of TCP, which enables faster web page responsiveness. VR applications demand more bandwidth to render intricate details of a virtual scene and will likely benefit from migrating to HTTP/3 powered by QUIC. 3. HTTPS (HyperText Transfer Protocol Secure) HTTPS extends HTTP and uses encryption for secure communications. 4. WebSocket WebSocket is a protocol that provides full-duplex communications over TCP. Clients establish WebSockets to receive real-time updates from the back-end services. Unlike REST, which always “pulls” data, WebSocket enables data to be “pushed”. Applications, like online gaming, stock trading, and messaging apps leverage WebSocket for real-time communication. 5. TCP (Transmission Control Protocol) TCP is is designed to send packets across the internet and ensure the successful delivery of data and messages over networks. Many application-layer protocols build on top of TCP. 6. UDP (User Datagram Protocol) UDP sends packets directly to a target computer, without establishing a connection first. UDP is commonly used in time-sensitive communications where occasionally dropping packets is better than waiting. Voice and video traffic are often sent using this protocol. 7. SMTP (Simple Mail Transfer Protocol) SMTP is a standard protocol to transfer electronic mail from one user to another. 8. FTP (File Transfer Protocol) FTP is used to transfer computer files between client and server. It has separate connections for the control channel and data channel. – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

A Visual Guide to CI/CD 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 (𝗖𝗜) is a foundational practice in DevOps where developers frequently merge code changes into the main branch, often multiple times a day. This process is complemented by automated testing to ensure new changes integrate seamlessly with existing code. The primary goal of CI is to find and address bugs quicker, improve software quality, and reduce the time required to validate and release updates 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 (𝗖𝗗) automates deploying code changes to a production without human intervention. It ensures every change passing all automated tests gets deployed. It accelerates customer feedback by releasing updates more frequently. CD also reduces pressure on developers by eliminating manual release processes. Some companies rely on 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗗𝗲𝗹𝗶𝘃𝗲𝗿𝘆 instead. Continuous Delivery extends CI by automatically preparing code changes for release to production. However, unlike Continuous Deployment, it requires manual approval prior to production deployment. This practice ensures that all changes are automatically built, tested, and ready for release. It allows teams to deploy new changes anytime at the push of a button. – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/496keA7

The Ultimate Software Architect Knowledge Map

A Crash Course on Scaling the Data Layer

Data Pipelines Overview.

Who are the Fantastic Four of System Design? Scalability, Availability, Reliability, and Performance. They are the most critical components to crafting successful software systems. Let’s look at each of them with implementation techniques: 1 - Scalability Scalability ensures that your application can handle more load without compromising performance. 2 - Availability Availability makes sure that your application is always ready to serve the users and downtime is minimal. 3 - Reliability Reliability is about building software that consistently delivers correct results. 4 - Performance Performance is the ability of a system to carry out its tasks at an expected rate under peak load using available resources. Over to you: What are the other pillars of system design and strategies you’ve come across? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

What does ACID mean?

Data Scientist Roadmap #Infographic by @DataScienceDojo

What does an API gateway do? An API gateway acts as a single entry point for clients, handling request routing, response composition, and protocol translation. It simplifies client interactions with microservices and offers additional features like rate limiting, authentication, monitoring, and more. To better understand how an API gateway works, let's look at how it processes a request: 1) Initial request handling Client requests are sent to the API gateway, which acts as the entry point for all incoming API traffic, rather than directly accessing the backend services. 2) Request validation The API gateway processes and validates the request’s attributes to ensure it’s correctly formatted. 3) Security checks It then performs checks against allow-lists and deny-lists to filter out unauthorized or harmful requests. 4) Authentication and authorization The API gateway validates the request, checking for proper authentication (e.g., verifying tokens or credentials) and ensuring the client has the necessary permissions (authorization) to access the requested resources. 5) Rate limiting Rate limiting rules are enforced; if the request exceeds the allowed limit, it’s rejected. 6) Service discovery and routing Once passing basic checks, the API gateway then finds the relevant service to route the request to by matching the path. 7) Protocol translation The API gateway transforms the request into the appropriate protocol and sends it to the service. 8) Response aggregation If the request requires data from multiple services, the API gateway aggregates the responses. It sends requests to each relevant service, collects the results, and composes them into a single, cohesive response. 9) Response delivery The gateway sends the processed response back to the client, ensuring it’s delivered in the expected format and within an optimal timeframe. 10) Logging, monitoring, fault handling, and caching Throughout this process, the API gateway logs each request and response and monitors key metrics such as latency, error rates, and throughput. These logs and metrics help in troubleshooting, scaling, and optimizing the system. It also deals with faults (circuit break), and provides response caching. An API gateway is a powerful tool that not only simplifies client interactions with microservices but also enhances security, performance, and reliability through comprehensive request processing and monitoring. ~~ Thank you to our partner Postman who keeps our content free to the community. How can we automate the health and performance of our APIs? Postman Monitors is a great way to do this. Learn more about it here: drp.li/postman-monito…

API Gateway vs Load Balancer — What's the Difference? An API Gateway acts as a single entry point for clients, handling request routing, composition, and protocol translation. It simplifies client interactions with microservices and offers features like rate limiting, authentication, and monitoring. Load balancers are concerned with routing client requests across multiple servers to distribute load and prevent bottlenecks. This helps maximize throughput, reduce response time, and optimize resource use. So what are the key differences? API gateways focus on request management and microservice communication, while Load Balancers focus on traffic distribution and server load management. API gateways operate at the application layer (L7), while Load Balancers can operate at both transport (L4) or application (L7) layers. API gateways offer features like routing, rate limiting, authentication, service discovery, parameter validation, circuit breakers, and more. Load Balancers handle traffic distribution and failover. An API gateway is Ideal for microservice architectures needing centralized API request management. A load balancer is essential for applications requiring high availability, distributing traffic across multiple servers. In summary, API Gateways manage and secure API calls, while Load Balancers ensure efficient traffic distribution. 💭 I’d love to hear your thoughts. What would you like to add? What topics would you like me to cover? 💬 ~~ Thank you to our partner Postman who keeps our content free to the community. Do you have team members who are blocked because they're waiting for you to ship a new API? Unblock them using mock servers. Learn more here: drp.li/postman-mock-s…

10 Key Data Structures We Use Every Day

How do we design secure and safe APIs? The rise in API-related security breaches highlights the necessity for robust API security. Let’s look at 12 essential tips for improving API security: 🔹HTTPS Enforcing HTTPS for all API connections is a critical step in securing sensitive data since it ensures data encryption in transit, preventing attacks and interceptions. 🔹 Rate limiting and throttling Throttling and rate limiting are vital for reducing API abuse and protecting against DDoS attacks as they manage request rates, which keeps our API available for legitimate users. 🔹 Authentication Authentication is another must-have. Leverage strong authentication mechanisms, such as OAuth, to verify user or system identities. Learn more about API authentication here: drp.li/api-authentica… 🔹 Authorization After authentication comes authorization. Follow the least privilege principle to ensure users access only role-relevant data and actions, reducing unauthorized access risks. 🔹 Input validation Validating API inputs is crucial to safeguard against vulnerabilities like SQL injection and XSS. Whitelisting can also be useful here to ensure only valid data is processed. 🔹 API gateway Deploy an API Gateway as a security layer, managing authentication, monitoring traffic, and enforcing policies like rate limits. Learn more about API gateways here: drp.li/api-gateways-e… 🔹 Regular security audits Regular security audits and penetration testing are advisable to identify and fix vulnerabilities, preventing exploitation and maintaining API security. 🔹Dependency management Regularly updating software dependencies is important to mitigate risks from vulnerabilities in external libraries. 🔹 Logging and monitoring Investing in comprehensive logging and real-time monitoring is vital for early detection of suspicious activities, enabling swift incident response to mitigate security breaches. 🔹API versioning To manage changes and updates securely, utilize proper API versioning, which prevents compatibility and security issues. Learn more about API versioning here: drp.li/api-encryption… 🔹Data encryption at rest Encrypting sensitive data at rest is crucial to prevent unauthorized access and comply with data protection regulations. API security isn’t a nice to have; it’s a must. Following the techniques and best practices I’ve shared above will take you a long way, they are the foundations of building safe and secure APIs. A great way to automate checking the health, performance, and security of your APIs is Postman Monitors. Learn more about it here: drp.li/postman-monito…

100X Postgres Scaling at Figma. With 3 million monthly users, Figma’s user base has increased by 200% since 2018. As a result, its Postgres database witnessed a whopping 100X growth. 1 - Vertical Scaling and Replication Figma used a single, large Amazon RDS database. As a first step, they upgraded to the largest instance available (from r5.12xlarge to r5.24xlarge). They also created multiple read replicas to scale read traffic and added PgBouncer as a connection pooler to limit the impact of a growing number of connections. 2 - Vertical Partitioning The next step was vertical partitioning. They migrated high-traffic tables like “Figma Files” and “Organizations” into their separate databases. Multiple PgBouncer instances were used to manage the connections for these separate databases. 3 - Horizontal Partitioning Over time, some tables crossed several terabytes of data and billions of rows. Postgres Vacuum became an issue and max IOPS exceeded the limits of Amazon RDS at the time. To solve this, Figma implemented horizontal partitioning by splitting large tables across multiple physical databases. A new DBProxy service was built to handle routing and query execution. Over to you - Would you have done something differently? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

What do Amazon, Netflix, and Uber have in common? They are extremely good at scaling their system whenever needed. Here are 8 must-know strategies to scale your system. 1 - Stateless Services Design stateless services because they don’t rely on server-specific data and are easier to scale. 2 - Horizontal Scaling Add more servers so that the workload can be shared. 3 - Load Balancing Use a load balancer to distribute incoming requests evenly across multiple servers. 4 - Auto Scaling Implement auto-scaling policies to adjust resources based on real-time traffic. 5 - Caching Use caching to reduce the load on the database and handle repetitive requests at scale. 6 - Database Replication Replicate data across multiple nodes to scale the read operations while improving redundancy. 7 - Database Sharding Distribute data across multiple instances to scale the writes as well as reads. 8 - Async Processing Move time-consuming and resource-intensive tasks to background workers using async processing to scale out new requests. Over to you: Which other strategies have you used? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

Best ways to test system functionality. Testing system functionality is a crucial step in software development and engineering processes. It ensures that a system or software application performs as expected, meets user requirements, and operates reliably. Here we delve into the best ways: 1. Unit Testing: Ensures individual code components work correctly in isolation. 2. Integration Testing: Verifies that different system parts function seamlessly together. 3. System Testing: Assesses the entire system's compliance with user requirements and performance. 4. Load Testing: Tests a system's ability to handle high workloads and identifies performance issues. 5. Error Testing: Evaluates how the software handles invalid inputs and error conditions. 6. Test Automation: Automates test case execution for efficiency, repeatability, and error reduction. Over to you: How do you approach testing system functionality in your software development or engineering projects? Over to you: what's your company's release process look like? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

Load balancing algorithms developers should know. Effective load balancing is crucial in system design, providing high availability and optimizing resource utilization. Let's look at how some of the most popular load balancing algorithms work. 🔹 Static Algorithms 1) Round robin It distributes requests sequentially between servers, ensuring equitable distribution. Despite its simplicity, it does not account for server load, which might be a drawback when demand changes significantly. 2) Random Implements a simple way of distributing requests regardless of server load or capability. This form of load distribution is basic, less precise, and suitable for less complicated applications. 3) IP hash Uses a consistent hashing method depending on the client's IP address to route requests. This technique is one way to ensure session persistence by consistently directing requests from the same client to the same server. 4) Weighted round robin Improves round robin by assigning requests based on server capacity, aiming to better utilize resources by allocating more requests to higher-capacity servers. This approach seeks to optimize resource use, though actual results can vary with request complexity and system conditions. 🔹 Dynamic Algorithms 5) Least connections Intelligently sends requests to the server with the fewest active connections, adapting to changing loads. This technique aims to better reflect current server utilization, potentially leading to more efficient resource consumption. 6) Least response time Targets performance by routing requests to the server with the quickest response time. By considering both current server load and performance, this technique supports faster processing, potentially reducing response times for users. While these are some of the most popular load-balancing strategies, there are other algorithms that also address specific needs and challenges. Choosing the right algorithm is very important to ensuring your application remains scalable, reliable, and efficient. ~~ A big thank you to our partner Postman who keeps our content free to the community. Did you know there is a VS Code extension for Postman? Check it out: drp.li/postman-vscode…

TCP/IP Model

What is MQTT, how does it work, and why is it becoming increasingly more important? MQTT was developed in the late 90s by IBM. It was used in specific industries that required lightweight communication. But it’s widespread adoption occurred with the rise of the Internet of Things (IoT) landscape. MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol optimized for high-latency or unreliable networks. Its minimal bandwidth and resource consumption, along with support for real-time data transmission, among other qualities, have made it ideal for IoT environments. From simple sensors to complex industrial machines, MQTT facilitates efficient data exchange, even in unreliable network conditions. MQTT operates on top of TCP/IP. It's focused on simple, efficient, and reliable message exchange. Below are the core concepts of MQTT. 🔹 Broker — this central hub ensures that messages reach their intended destinations. 🔹 Client — any MQTT-enabled device capable of publishing and receiving messages. 🔹 Topic — a defined category for publishing and subscribing to messages that makes communication organized. 🔹 Quality of service — categorizes the guarantee level for message delivery based on its urgency. 🔹 Message — the actual data being transmitted between clients and the broker. 🔹 Session — the state maintained between a client and a broker, including subscriptions and undelivered messages. The MQTT communication flow is quite straightforward: 1) Clients connect to the MQTT broker and subscribe to topics of interest. 2) When a client publishes a message to a topic, the broker receives and processes it. 3) The broker then forwards the message to every client who has subscribed to that topic. This streamlined process, together with the protocol's lightweight nature, makes MQTT ideal for IoT scenarios with limited connectivity or stringent power constraints. MQTT ensures efficient data transmission and offers reliable message delivery across various network conditions. This reliability is complemented by support for multiple QoS levels of diverse requirements. These features enable real-time monitoring and control and improve the scalability and responsiveness of IoT systems. With its reliable, scalable, and resource-conscious messaging solution, MQTT forms the backbone of IoT communication. ~~ A big thank you to our partner Postman who keeps our content free to the community. Wish building APIs was simpler and more organized? Postman Collections does just that. Check it out: drp.li/postman-collec…