API Design and Networking

HTTP is the set of rules a web browser and a server use to pass notes. With the older version, HTTP/1.1, one connection (one open road between two computers) carries one request and its answer at a time. If a big picture is coming down the road, a small note behind it has to wait. Browsers worked around this by opening about six roads to each website, which costs extra time and work.

HTTP/2 uses one road for everything. Every answer is cut into small pieces, and each piece gets a label saying which request it belongs to. One request and its answer together are called a stream. Pieces from many streams are mixed together on the same road, and the other side sorts them back out using the labels. This is called multiplexing. A big picture no longer blocks a small note, because their pieces take turns.

HTTP/2 also squeezes the headers, the little address labels that ride along with every request. Many of them repeat on every note, like which browser you use and what kinds of answers you understand. So both sides keep a shared list and send a short number instead of the whole label. This is called header compression.

The weakness is the road underneath. HTTP/2 runs on TCP, a delivery rule that promises every piece arrives in the exact order it was sent. Picture one conveyor belt carrying everyone's toy pieces. If one piece falls off, the whole belt stops until that piece is sent again, even for kids whose pieces were fine. This is called head-of-line blocking, and it hurts most on bumpy networks like a weak phone signal.

HTTP/3 keeps the good ideas of HTTP/2 but changes the road underneath. Instead of TCP, it uses newer rules called QUIC, which are built on UDP. UDP is a very plain way to send pieces, with no promise about order or arrival. QUIC adds its own promises on top, but it keeps track of order for each stream separately.

Picture a truck carrying several small belts, one for each kid. If a piece falls off one belt, only that kid waits for a new copy. Everyone else keeps getting their pieces. So a lost piece only slows down its own stream.

QUIC is also quicker to start. Before two computers can talk safely, they say hello and agree on a secret code for locking their messages, called encryption. TCP with modern encryption needs two back and forth trips before the first real request. QUIC does both in one trip. When you come back to a site you visited before, it can even send the first request with zero waiting trips. A sneaky listener could copy and replay that very first early request, so it is only used for safe things like reading a page.

QUIC gives each conversation its own ID number instead of tying it to the phone's network address. So when you walk out of the house and your phone switches from Wi-Fi to mobile data, the conversation keeps going instead of starting over. The trade-offs: some office and school networks block or slow down UDP, so browsers keep HTTP/2 as a backup, and QUIC does more of its work in ordinary programs, which can cost servers more computing power.

Imagine a pizza shop where you must order with a printed form. The form lists every box you can fill in and what goes in each box: a number for the size, a word for the topping. You and the shop got the very same form ahead of time, so nobody misreads an order. gRPC works like this. The form is a file written with Protocol Buffers (often called protobuf), and it describes every message and every question one service can ask another.

From that one form, tools write the calling code for you, in many programming languages. Asking another service then looks like asking a friend down the hall a normal question. The generated code packs each message into compact binary, a tight row of numbers instead of readable words. That is smaller and quicker to pack and unpack than JSON, the readable, labeled text most web APIs send.

gRPC rides on HTTP/2, so many calls share one connection. It also supports streaming: the server can send a long stream of answers to one question, the client can send a stream of messages, or both can talk at the same time. Each call can carry a deadline, so a slow answer is given up on instead of waited for forever.

It shines when services inside one company talk to each other, because everyone can share the form. The trade-offs: binary messages are hard for people to read while hunting bugs, and web browsers cannot speak gRPC directly, so they need a helper called gRPC-Web plus a small translator in front of the server. Also, since one connection stays open for a long time, the load balancer must spread out single calls, not just connections, or one server ends up doing all the work.

A normal web request is like calling, asking one question, and hanging up. If you want to know when a new chat message arrives, you have to keep calling back: anything new? anything new? This is called polling. It is like a kid in the back seat asking are we there yet every minute. Most answers are no, and every question costs a trip.

A WebSocket starts as a normal HTTP request that says: can we switch this into a phone call? If the server agrees, the same connection is upgraded into a long-lived, two-way line. Now either side can talk whenever it wants. The server can push a new message the moment it arrives, without being asked.

This is great for chat, multiplayer games, live sports scores and shared drawing boards, where small updates fly back and forth quickly. If only the server ever needs to talk, a simpler one-way stream called Server-Sent Events can be enough.

The costs: every open line holds a little memory on a server, and a popular app may hold millions of lines open at once. Lines drop when a phone loses signal, so apps need reconnect logic that tries again and catches up on anything missed. Load balancers must allow long connections and not hang up on quiet ones, so apps send tiny heartbeat messages (ping and pong) to show they are still there. And when friends are connected to different servers, a message must be passed between servers through a shared message hub.

A big app is often split into many small services, like classrooms in a big school: one for user accounts, one for orders, one for videos. Visitors should not wander the halls looking for the right room. Instead, everyone comes in through one front desk. That front desk is the API gateway.

The front desk does the shared chores once, so each classroom does not have to. It checks badges (authentication, making sure you are who you say you are). It sends you to the right classroom based on what you asked for (routing). It makes sure no visitor comes too often (rate limiting). It can gather answers from several classrooms into one reply, so a phone does not have to make five separate trips. And it writes down who came and went (logging).

The trade-offs: every request makes one extra stop, which adds a little time. Since everyone goes through the front desk, if it breaks, nobody gets in, so gateways run as several copies behind a load balancer. It is also tempting to stuff too much logic into the front desk, which turns it into a slow, crowded bottleneck. Keep the gateway for shared chores, and leave the real work to the services.

A load balancer is like the lunch line helper who sends each kid to the counter with the shortest line, so no single counter gets swamped. In computers, many copies of a server sit behind it, and it spreads requests among them. Common rules are taking turns (round robin) or picking the copy with the fewest people already waiting (least connections). It also stops sending anyone to a copy that has stopped answering its health checks.

The layer numbers come from a standard way of stacking network jobs into floors. A layer 4 load balancer only reads the outside of the envelope: the computer's address (the IP address) and the port, which is like an apartment number. It never opens the letter. That makes it very fast and simple, and it works for any kind of traffic, even locked (encrypted) traffic it cannot read. But it cannot tell what is being asked, so it cannot send video requests one way and shopping requests another.

A layer 7 load balancer opens the letter and reads the request itself: the path like /videos, the headers and the cookies. Now it can be clever. It can send /videos to the video team and /shop to the shop team, keep a user going to the same server using a cookie (sticky sessions), and retry a failed request somewhere else. It can also unlock the encryption itself, called ending TLS, so the servers behind it do not have to. The cost is more work for every single request, and it must hold the keys that unlock the traffic.

Many big systems use both: a layer 4 balancer at the very front to spread huge amounts of traffic cheaply, and layer 7 balancers behind it to make the smart choices.

If one visitor asks a million questions a second, everyone else gets slow answers, and the servers might fall over. Rate limiting is a hall monitor with a rule like: each kid may ask 10 questions a minute. When someone goes over, the server answers with HTTP status 429, which means too many requests, often with a Retry-After note saying how long to wait. Good clients wait and try again, leaving longer gaps each time. Limits are counted per user, per API key or per IP address, and when many servers share one limit, they keep the counts in a fast shared store such as Redis.

Token bucket: picture a jar of tokens. A helper drops in one token at a steady pace, say one each second, until the jar is full at 5. Each request must take a token, and if the jar is empty, the request is turned away. A full jar lets a quick burst of 5 through at once, but over time nobody goes faster than the drip. Leaky bucket: requests pour into a bucket with a small hole in the bottom. They drip out to the server one by one at a steady pace. If too many pour in, the bucket overflows and the extras are turned away. Bursts become a calm, even flow, but requests may wait in line.

Fixed window counter: count requests in each clock minute, and reset the count to zero when a new minute starts. It is simple and needs only one number per user. The weakness is the edge. With a limit of 10, a kid can ask 10 times at 0:59 and 10 more at 1:00, which is 20 in about two seconds.

Sliding window log: write down the time of every request, and count only the ones from the last 60 seconds. It is exact, but remembering every time stamp uses a lot of memory for busy users. Sliding window counter: keep just two counts, last minute and this minute, and blend them. If we are 30 percent of the way into this minute, count 70 percent of last minute's requests plus all of this minute's. It assumes last minute's requests were spread evenly, so it is a close guess, not exact, but it needs very little memory.

• Token bucket: allows short bursts, keeps a steady average.
• Leaky bucket: turns bursts into an even flow.
• Fixed window counter: simplest, but lets a double burst through at the edge.
• Sliding window log: exact, but hungry for memory.
• Sliding window counter: a good estimate with tiny memory.