Reverse Proxy in System Design: How NGINX, Cloudflare and Load Balancers Work
Every time you visit a popular website, your request doesn't go straight to the server that actually runs the application. Something intercepts it first — inspects it, secures it, maybe serves it from cache, then forwards it to the right backend server.
That invisible middleman is the Reverse Proxy. It is one of the most fundamental building blocks of modern web infrastructure. NGINX, Cloudflare, AWS Application Load Balancer, Traefik, HAProxy — all of these are reverse proxies (or built on the concept).
Understanding reverse proxies is not optional for any engineer who builds systems that serve real users. Let's master it completely — from the first analogy to production-grade configuration.
🌐 NGINX — Powers over 34% of all websites on Earth
☁️ Cloudflare — A reverse proxy handling 50M+ requests/second
☁️ AWS ALB / NLB — Amazon's managed reverse proxy service
🔵 HAProxy — The backbone of Twitter, GitHub, Reddit
🐳 Traefik — The cloud-native reverse proxy for Kubernetes
🔶 Envoy — The proxy engine inside every service mesh (Istio, Linkerd)
📱 Netflix, Uber, Airbnb — All use reverse proxies as the first layer of their architecture
If you have ever used the internet, a reverse proxy handled your request.
🏨 Section 1: Think of a Reverse Proxy Like a Hotel Concierge
Imagine a grand hotel with hundreds of rooms, restaurants, a spa, a gym, and a rooftop bar. Guests don't walk directly to any of these places. They go to the front desk concierge first.
The concierge checks who you are (authentication), tells you what's available (routing), calls ahead to confirm the room/restaurant is ready (health checking), gives you a translated guide if you need it (protocol translation), and directs you to the right place (load balancing between 5 different restaurants).
You never need to know how many kitchens there are, which chef is on duty, or which restaurant is full. The concierge handles all of that. Your backend servers are like the hotel's internal operations — the guests (clients) never see or contact them directly.
| 🏨 Hotel Concierge | 🛡️ Reverse Proxy Reality | 🔧 Engineering Concept |
|---|---|---|
| Checks guest ID before entry | Validates auth tokens, API keys | Authentication offloading |
| Routes guests to the right floor/room | Routes /api → App Server, /static → CDN | Path-based routing |
| Spreads guests across 5 restaurants | Distributes requests across servers | Load balancing |
| Doesn't send you to a closed restaurant | Doesn't route to unhealthy servers | Health checking |
| Guests never see the kitchen staff | Clients never see backend server IPs | IP masking / anonymity |
| Translates for foreign guests | Translates HTTPS → HTTP internally | SSL/TLS termination |
🔄 Section 2: Forward Proxy vs Reverse Proxy — The Critical Distinction
This is the most common point of confusion, even for experienced engineers. The difference is about who is being hidden from whom.
📤 Forward Proxy — Hides the CLIENT
Sits in front of clients. The server sees the proxy's IP, not the client's real IP. Used for: VPNs, corporate internet filtering, anonymity (Tor), bypassing geo-restrictions.
PROXY
Server doesn't know who the real client is.
📥 Reverse Proxy — Hides the SERVER
Sits in front of servers. The client sees the proxy's IP, not the backend server's IP. Used for: load balancing, security, caching, SSL termination — everything we discuss in this post!
PROXY
(hidden)
(hidden)
Client doesn't know which real server handled the request.
Forward Proxy = Faces forward toward the internet for the client. Client's agent in the outside world.
Reverse Proxy = Faces backward toward your private servers. Server's guardian facing the internet.
Think of it like this: you hire a lawyer (forward proxy) to represent you in court. A company hires a PR spokesperson (reverse proxy) to represent them to the public. The lawyer speaks FOR you. The spokesperson speaks FOR the company.
🗺️ Section 3: The Big Picture — What a Reverse Proxy Does
A reverse proxy is not a single feature — it is a platform that provides many capabilities simultaneously. Every request passes through a pipeline of functions before reaching your backend servers.
🛡️ Reverse Proxy — What Happens to Every Request
Sends HTTPS request
App Service
:8080
App Service
:8080
Admin Service
:9090
CDN / S3
:443
↑ Clients only ever talk to the proxy. They never know how many backend servers exist or where they are.
🔒 Section 4: SSL/TLS Termination — One Lock for Everything
HTTPS encryption (TLS) is computationally expensive. Performing the TLS handshake on every backend server would require each server to hold SSL certificates, manage private keys, and spend CPU on cryptography. The reverse proxy makes this radically simpler.
At an airport, security happens at ONE central checkpoint — not at the door of every gate. Once you're through security (TLS terminated at the reverse proxy), you move freely between gates (backend servers) without being re-screened.
The reverse proxy decrypts the HTTPS request from the client, then forwards a plain HTTP request to backend servers on the internal network. Your backend servers don't even need to know about SSL!
🔒 SSL Termination — Two Separate Connections
HTTPS request
🔐 Encrypted
Terminates TLS
Holds the certificate
🔑 Has private key
Plain HTTP
No SSL needed!
✅ Simpler
🔹 Certificate lives in ONE place — easy to renew
🔹 Backend servers are simpler — no SSL config
🔹 TLS overhead on proxy (usually faster hardware)
🔹 Internal traffic stays on trusted private network
🔹 Highly regulated environments (banking, healthcare)
🔹 Zero-trust architectures
Solution: SSL Passthrough (proxy doesn't decrypt) or
re-encryption (proxy re-encrypts to backend)
⚖️ Section 5: Load Balancing — Spreading the Work Fairly
One of the most critical functions of a reverse proxy is distributing incoming traffic across multiple backend servers so no single server gets overwhelmed. Reverse proxies support several load balancing strategies.
Requests go to servers in a fixed cycle: Server 1 → Server 2 → Server 3 → Server 1 → ... Simple, fair, and works well when all servers have equal capacity and each request takes similar time. Best for: Stateless services where all servers are identical.
New requests go to the server with the fewest active connections right now. Much smarter for variable-length requests (some requests take 10ms, some take 5 seconds). Best for: Long-lived connections (WebSockets, streaming, file uploads).
Hash the client's IP address → always route to the same backend server. Same user always lands on the same server — useful for applications that store session data in server memory (though stateless design is better). Best for: Legacy stateful apps that can't be redesigned.
Assign weights to servers. Server with weight=3 gets 3x more traffic than weight=1. Use when servers have different hardware capacities (one has 64GB RAM, another has 16GB). Best for: Heterogeneous server fleets.
Route to the server with the lowest average response time AND fewest connections. The most intelligent algorithm — used by HAProxy and Envoy. Adapts automatically to server performance in real time. Best for: Latency-sensitive APIs where response time matters most.
💓 Health Checking — Never Route to a Dead Server
Reverse proxies continuously probe backend servers. If a server stops responding, it's removed from rotation instantly. Traffic flows to healthy servers. Users experience nothing.
💓 Health Check Animation — Proxy Pinging Backend Servers
Proxy
(pings every 10s)
↑ Server 3 is removed the moment health checks fail. All traffic goes to Server 1 and Server 2. No human needed. No outage for users. Automatic recovery when Server 3 comes back!
🗺️ Section 6: Request Routing — Sending Requests to the Right Place
A reverse proxy can route different requests to completely different backend services based on the URL path, hostname, HTTP headers, or cookies. This is what enables microservices architectures to appear as a single unified API.
🗺️ Path-Based Routing — One Domain, Many Services
:8080
:9001
Edge Cache
:9090 (IP whitelist)
Servers :8081
The client sends everything to one URL. The reverse proxy silently routes to the right microservice. This is how companies run dozens of microservices behind a single domain!
🏷️ Host-Based Routing (Virtual Hosts)
A reverse proxy can also route based on the domain name in the request. One server, many domains — each routed to a different backend.
api.company.com
www.company.com
admin.company.com
docs.company.com
↑ All four domains resolve to the same public IP (the reverse proxy's IP).
The proxy reads the Host header and routes to the right service. Pure magic!
🆚 Section 7: Reverse Proxy vs API Gateway vs Load Balancer
These three terms are often used interchangeably — and incorrectly. Let's clarify exactly what each one is and when to use which.
| Feature | ⚖️ Load Balancer | 🛡️ Reverse Proxy | 🚪 API Gateway |
|---|---|---|---|
| Primary Job | Distribute traffic | Intercept + transform + forward | Manage API lifecycle |
| OSI Layer | L4 (TCP) or L7 (HTTP) | L7 (HTTP/HTTPS) | L7 (HTTP/HTTPS) |
| SSL Termination | ⚠️ Sometimes (L7 only) | ✅ Yes | ✅ Yes |
| Load Balancing | ✅ Core function | ✅ Yes | ✅ Yes |
| URL Routing | ❌ L4 only, ⚠️ L7 | ✅ Yes | ✅ Yes |
| Auth / API Keys | ❌ No | ⚠️ Basic | ✅ Core feature |
| Rate Limiting | ❌ No | ⚠️ Basic | ✅ Advanced |
| Caching | ❌ No | ✅ Yes | ⚠️ Sometimes |
| Examples | AWS NLB, HAProxy L4 | NGINX, HAProxy L7, Traefik | AWS API GW, Kong, Apigee |
🔹 Load Balancer = Simple traffic distributor. Knows nothing about HTTP content.
🔹 Reverse Proxy = Smart HTTP intermediary. Reads, transforms, caches, routes.
🔹 API Gateway = Reverse proxy + API management (auth, quotas, analytics, versioning).
In practice: every API Gateway is a reverse proxy. Most reverse proxies can act as load balancers. But a simple load balancer is NOT a reverse proxy. The hierarchy goes: Load Balancer ⊂ Reverse Proxy ⊂ API Gateway.
📦 Section 8: Caching and Compression — Speed Boosters
📦 Edge Caching at the Proxy
A reverse proxy can cache responses from backend servers. If the same resource is requested frequently, the proxy serves it directly without forwarding to the backend — saving backend CPU and dramatically reducing latency.
📦 Cache HIT vs MISS — Same as CDN, but Closer
🗜️ Response Compression (Gzip)
The reverse proxy can compress responses before sending to clients. A 200KB HTML page compresses to ~20KB with gzip — 10x smaller! This saves bandwidth and makes pages load faster for users on slow connections.
200 KB
(plain HTML)
Gzip compress
~20 KB
10x smaller! 🚀
Client sends Accept-Encoding: gzip → Proxy compresses response → Client decompresses.
Transparent to both client and backend. Pure performance win.
🛠️ Section 9: Popular Reverse Proxies — Which One to Choose
The most popular reverse proxy in the world. Powers ~34% of all websites. Extremely high performance (event-driven architecture, handles 10,000+ concurrent connections). Works as: reverse proxy, load balancer, static file server, cache, and web server. Configuration is declarative (nginx.conf files).
Used by: Dropbox, Netflix, WordPress.com, GitHub
Built specifically for load balancing and high availability. Extremely feature-rich: ACL rules, health checking, session stickiness, advanced routing. Supports both Layer 4 (TCP) and Layer 7 (HTTP) proxying. Battle-hardened at extreme scale — handles millions of connections per second.
Used by: Twitter, GitHub, Reddit, Airbnb, Instagram
Designed for containerised, cloud-native environments (Docker, Kubernetes). Auto-discovers services — when you spin up a new container, Traefik automatically configures routing without any manual config changes. Integrates with Let's Encrypt for automatic SSL. Perfect for microservices.
Used by: BNP Paribas, Michelin, Zalando
Originally built by Lyft. Now the most popular service mesh proxy. Powers Istio, AWS App Mesh, and Google Traffic Director. Extremely powerful observability (metrics, tracing, logging for every request). Uses gRPC for management plane communication. Dynamically configurable at runtime.
Used by: Lyft, AWS (App Mesh), Google Cloud, Stripe
AWS Application Load Balancer — a fully managed Layer 7 reverse proxy and load balancer. No server to manage. Auto-scales. Native integration with EC2, ECS, Lambda, Cognito. Supports path-based routing, host-based routing, sticky sessions, WebSocket, HTTP/2. Cost: pay per LCU (Load Capacity Unit) — can get expensive at high scale.
📡 Section 10: Layer 4 vs Layer 7 Proxies — Two Different Animals
📦 Layer 4 Proxy (Transport Layer)
Works at the TCP/UDP level. It doesn't understand HTTP — it just sees raw packets. Cannot read URL paths, headers, or cookies.
Examples: AWS NLB, HAProxy TCP mode, LVS
🌐 Layer 7 Proxy (Application Layer)
Works at the HTTP/HTTPS level. Reads and understands the full request content: URL, headers, body, cookies.
Examples: NGINX, HAProxy HTTP mode, AWS ALB, Traefik, Envoy
🔹 Use L4 when: extremely high throughput (millions of connections/sec), non-HTTP protocols (MySQL, Redis, SMTP), or when you need the absolute lowest latency.
🔹 Use L7 when: you need URL routing, SSL termination, HTTP-specific features, caching, or security inspection. This is the choice for 95% of web applications.
Many production setups use BOTH: an L4 proxy at the network edge for raw throughput, with L7 proxies behind it for smart HTTP routing. AWS does this with NLB (L4) → ALB (L7).
🗺️ Section 11: Everything Together — Complete Reverse Proxy Architecture
🛡️ Reverse Proxy — Complete Production Architecture
:8080
:8080
:9001
:8081
:9090
(only backends reach here)
📐 Section 12: Core Design Principles from Reverse Proxies
The reverse proxy gives you one place to implement security, logging, caching, and rate limiting — instead of reimplementing them in every microservice. This is the Single Responsibility Principle at the infrastructure level. Backend services should focus only on business logic. Everything else belongs at the edge.
Backend servers should live on a private network, accessible only through the proxy. This hides your infrastructure topology from attackers. Even if someone discovers a vulnerability in your app, they can't directly attack the server — they must go through the proxy's security layer first.
Every backend server must expose a health check endpoint (e.g., GET /health).
The proxy polls this endpoint and automatically removes unhealthy servers.
Without health checks, the proxy will continue sending traffic to dead servers,
causing errors for users. Health checks are not optional — they are mandatory.
Ironic truth: the reverse proxy is supposed to provide high availability — but if it is a single server, it becomes the single point of failure! Production setups use HA (High Availability) proxy pairs: two NGINX servers with a shared virtual IP (using Keepalived/VRRP). If the primary proxy fails, the backup takes over the virtual IP automatically. Zero downtime.
The reverse proxy sees every single request — making it the perfect place for observability. Log every request (URL, status, latency, user agent). Emit metrics (requests/sec, error rate, p99 latency). This gives you a complete view of your system's health without instrumenting every individual service. NGINX + Prometheus exporter is a powerful combo.
🎓 Section 13: Cheat Sheet
Reverse proxy questions appear in almost every system design interview. Here is exactly how to handle every variant:
Answer: A reverse proxy sits in front of backend servers, intercepting all incoming client requests. It provides: SSL termination, load balancing, caching, compression, request routing, security (WAF, rate limiting), and health checking. Clients only ever communicate with the proxy — backend servers are never exposed to the internet. The proxy hides the server identity (reverse proxy) vs. a forward proxy which hides the client identity.
Answer: SSL termination at the reverse proxy (NGINX/ALB). The proxy holds the SSL certificate and private key. Clients connect over HTTPS. Proxy decrypts and forwards plain HTTP to backend servers (which stay on a private internal network). Benefits: one certificate to manage, backend servers stay simple, TLS computation done once at the edge.
Answer: Path-based routing at the reverse proxy.
/api/* → API service. /auth/* → Auth service. /static/* → CDN.
The client sees one domain. The proxy reads the URL path and routes to the correct backend.
Each backend service runs on a different internal port or private IP.
Configure using NGINX location blocks or HAProxy ACL rules.
Answer: Depends on the workload. Round-robin: simple, equal servers, stateless APIs. Least Connections: best for variable-length requests (file uploads, long polling). IP Hash: required if session affinity is needed (legacy stateful apps). Weighted Round-Robin: when servers have different hardware capacity. For most modern stateless microservices → Least Connections or Round-Robin.
Answer: Load balancer = primarily distributes traffic. May be L4 (no HTTP understanding) or L7. Reverse proxy = L7 intermediary: reads HTTP content, routes, caches, compresses, terminates SSL. Every reverse proxy can load balance. Not every load balancer is a reverse proxy. API gateway = reverse proxy + API management: auth, rate limiting, API versioning, analytics dashboards, API keys. Kong, AWS API Gateway, Apigee are examples. They're a spectrum of increasing capability: LB ⊂ Reverse Proxy ⊂ API Gateway.
Answer: Run two NGINX/HAProxy instances (primary + standby) on separate machines. Use Keepalived/VRRP to share a virtual IP between them. DNS points to the virtual IP. If primary fails, standby takes over the virtual IP in seconds. For cloud: use managed services (AWS ALB) which are inherently HA and auto-scale. Add health checks to all backends. Monitor proxy metrics. Ensure proxy itself is not the single point of failure.
🎉 Final Summary
The reverse proxy teaches a universal engineering principle: centralise cross-cutting concerns at the edge, keep backend services focused on business logic.
Security, caching, compression, SSL, rate limiting, routing — none of these should be duplicated in every microservice. They belong at the edge, implemented once, applied universally. When you put the reverse proxy in your architecture, every service behind it instantly gets all these capabilities for free.
That is the power of the single entry point design pattern. And it's why the reverse proxy is one of the most important building blocks in all of software architecture. 🏛️
Happy Learning! Keep Building! 🔥
Comments
Post a Comment