You know those tiny links like
https://bit.ly/3xKpQ9 that magically take you to a giant website?
By the end of this post, you will understand every single step of what happens —
from the moment you click that tiny link to the moment the full website appears on your screen!
💡 Did You Know?
Bitly alone processes over 10 billion clicks per month! That means their system must handle thousands of redirects every single second — without slowing down or crashing. Building such a system is one of the most popular System Design interview questions at companies like Google, Amazon, and Meta. 🌍
Bitly alone processes over 10 billion clicks per month! That means their system must handle thousands of redirects every single second — without slowing down or crashing. Building such a system is one of the most popular System Design interview questions at companies like Google, Amazon, and Meta. 🌍
📦 What is a URL Shortener? (The Pizza Box Analogy)
Imagine you ordered a giant pizza 🍕. The full address of the pizza shop is:
"Pizza Palace, 3rd Floor, Building No. 47, Green Park Avenue, Near Central Metro Station, New Delhi - 110016, India"
That is way too long to share with a friend over a text message! So you just say: "Meet me at Pizza Palace — you know the one!"
A URL Shortener does exactly this for web links. It takes a super-long URL and gives it a tiny, shareable nickname that points to the same place.
BEFORE (Original Long URL):
┌─────────────────────────────────────────────────────────────────────┐
│ https://www.amazon.in/dp/B09G9HD6PD?ref=ppx_yo2ov_dt_b_product_ │
│ details&th=1&psc=1&tag=mysupercoolblog-21&linkCode=ogi │
└─────────────────────────────────────────────────────────────────────┘
⬇️ URL Shortener Magic ⬇️
AFTER (Short URL):
┌────────────────────────────┐
│ https://amzn.to/3Kp9Xz │
└────────────────────────────┘
✅ Simple Definition:
A URL Shortener is a service that converts a long, ugly web address into a short, clean link. When someone clicks the short link, they are automatically sent to the original long URL. This "automatic sending" is called a Redirect.
A URL Shortener is a service that converts a long, ugly web address into a short, clean link. When someone clicks the short link, they are automatically sent to the original long URL. This "automatic sending" is called a Redirect.
Why Do We Even Need URL Shorteners?
Great question! Let's think of real-life reasons why short URLs are super useful:
- 📱 Social Media Limits: Twitter (now X) used to have a 140-character limit. Long URLs would eat up all your space!
- 📊 Track Clicks: Businesses want to know — did anyone actually click my link? Short URLs come with built-in analytics.
- 💬 Easy to Share: Try reading a 200-character URL out loud over a phone call. Short URLs are much easier!
- 🖨️ Print & Billboards: You cannot put a 300-character URL on a physical poster. A short URL fits perfectly.
- 🔒 Hide Affiliate/Tracking Codes: Marketing teams use short URLs to hide messy tracking parameters from the end user.
🗺️ The Big Picture — What Happens When You Click a Short Link?
Let's walk through this step by step, like a treasure hunt! 🗺️
You type https://bit.ly/3xKp9 into your browser.
Here is exactly what happens next:
YOU (Browser) SHORT URL SERVER ORIGINAL WEBSITE
│ │ │
│ 1. GET /3xKp9 │ │
│──────────────────────────────────►│ │
│ │ │
│ │ 2. Lookup "3xKp9" in Database │
│ │ ───────────────────────── │
│ │ 3xKp9 → amazon.com/dp/... │
│ │ │
│ 3. HTTP 301/302 Redirect │ │
│◄──────────────────────────────────│ │
│ Location: amazon.com/dp/... │ │
│ │ │
│ 4. GET amazon.com/dp/... │ │
│──────────────────────────────────────────────────────────────────► │
│ │ │
│ 5. Here is your full webpage! ✅ │ │
│◄────────────────────────────────────────────────────────────────── │
💡 Think of it like a Post Office!
You write a letter to "Grandma, House No. 5". The post office knows that "House No. 5" is actually "42, Maple Street, Springfield, IL 62701". They look it up and forward your letter. The URL shortener is the post office! 📮
You write a letter to "Grandma, House No. 5". The post office knows that "House No. 5" is actually "42, Maple Street, Springfield, IL 62701". They look it up and forward your letter. The URL shortener is the post office! 📮
🏗️ Building Blocks — The 5 Core Components
Now let's open the hood and see the engine! Every URL shortener, whether it is a tiny side project or a billion-dollar service like Bitly, is built from these core building blocks:
┌─────────────────────────────────────────────────────────┐ │ URL SHORTENER SYSTEM COMPONENTS │ │ │ │ 1. 🌐 API Gateway / Load Balancer │ │ (The front door — takes all incoming requests) │ │ ⬇️ │ │ 2. ⚙️ Application Servers │ │ (The brain — runs the logic) │ │ ⬇️ │ │ 3. 💾 Database (SQL or NoSQL) │ │ (The notebook — stores short → long mappings) │ │ ⬇️ │ │ 4. ⚡ Cache (Redis / Memcached) │ │ (The quick-memory — stores popular links) │ │ ⬇️ │ │ 5. 📊 Analytics Service │ │ (The reporter — counts clicks, tracks location) │ └─────────────────────────────────────────────────────────┘
🔑 How Is the Short Code Generated? (The Most Important Part!)
This is the most interesting question! When you paste a long URL,
the system must create a unique short code like 3xKp9.
There are 3 main techniques used:
Technique 1: 🎲 Random String Generation
The simplest approach — just pick 6–8 random characters from the alphabet + numbers. Think of it like a lucky lottery ticket! The system picks a random combination and checks if it has already been used. If not, it uses that code.
💡 How many combinations are possible with 7 characters?
Using A–Z, a–z, and 0–9 (62 characters total):
62⁷ = 3,521,614,606,208 combinations — that is 3.5 trillion! You will never run out of unique short codes. 🎉
Using A–Z, a–z, and 0–9 (62 characters total):
62⁷ = 3,521,614,606,208 combinations — that is 3.5 trillion! You will never run out of unique short codes. 🎉
📝 What does this code do?
The function below creates a random 7-character short code using letters and numbers — just like picking random tiles from a Scrabble bag! Every time you call it, you get a different unique code like
The function below creates a random 7-character short code using letters and numbers — just like picking random tiles from a Scrabble bag! Every time you call it, you get a different unique code like
aB3xKp9.
# Python Example — Generating a Random Short Code
import random
import string
def generate_short_code(length=7):
"""
Creates a random short code like 'aB3xKp9'
using letters (A-Z, a-z) and numbers (0-9).
"""
characters = string.ascii_letters + string.digits # 62 possible characters
short_code = ''.join(random.choices(characters, k=length))
return short_code
# Let's try it!
print(generate_short_code()) # Output: aB3xKp9
print(generate_short_code()) # Output: mZ7qR2w
Technique 2: 🔢 Base62 Encoding of a Counter (Industry Standard!)
This is how most professional services work! Instead of random codes, the system uses a global counter. Every new URL gets the next number: 1, 2, 3, 4... Then it converts that number into a short Base62 string.
Counter Number → Base62 Code
──────────────────────────────
1 → "b"
62 → "ba"
3844 → "baa"
100,000 → "q0U"
1,000,000 → "4c92"
WHY BASE62?
Base 10 uses digits: 0–9 (10 symbols)
Base 16 (hex) uses: 0–9, A–F (16 symbols)
Base 62 uses: 0–9, A–Z, a–z (62 symbols)
→ More symbols = shorter string for the same number! 🎯
📝 What does this code do?
Think of Base62 like converting a big number into a secret language that uses letters AND numbers. The bigger the number, the longer the code — but it stays very short even for millions of URLs!
Think of Base62 like converting a big number into a secret language that uses letters AND numbers. The bigger the number, the longer the code — but it stays very short even for millions of URLs!
# Python Example — Base62 Encoding
BASE62_CHARS = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
def encode_base62(number):
"""
Converts a plain number (like 12345) into a Base62 short code.
Example: 12345 → "3d7"
"""
if number == 0:
return BASE62_CHARS[0]
result = []
while number > 0:
remainder = number % 62 # Find remainder
result.append(BASE62_CHARS[remainder]) # Map to character
number //= 62 # Reduce number
return ''.join(reversed(result)) # Reverse to get correct order
def decode_base62(short_code):
"""
Converts a Base62 short code back into the original number.
Example: "3d7" → 12345
"""
number = 0
for char in short_code:
number = number * 62 + BASE62_CHARS.index(char)
return number
# Let's test it!
print(encode_base62(1)) # Output: "1"
print(encode_base62(100000)) # Output: "q0U"
print(decode_base62("q0U")) # Output: 100000
Technique 3: 🔐 MD5/SHA Hash (Hash-Based Approach)
A hash function is like a fingerprint machine for data. You give it any long URL and it produces a fixed-length "fingerprint" of the URL. We then take just the first 7 characters of that fingerprint as our short code.
⚠️ Warning — Hash Collisions!
Two different URLs can sometimes produce the same first 7 characters of a hash. This is called a collision. In production systems, you must always check if the short code already exists before saving it!
Two different URLs can sometimes produce the same first 7 characters of a hash. This is called a collision. In production systems, you must always check if the short code already exists before saving it!
# Python Example — Hash-Based Short Code
import hashlib
def hash_based_short_code(long_url, length=7):
"""
Creates a short code by hashing the long URL.
The same URL always produces the same short code!
(This is called 'deterministic' behavior)
"""
# MD5 gives a 32-character hexadecimal string
hash_value = hashlib.md5(long_url.encode()).hexdigest()
# We only take the first 7 characters
return hash_value[:length]
# Test it!
url = "https://www.amazon.in/dp/B09G9HD6PD?ref=ppx_yo2ov"
print(hash_based_short_code(url)) # Always gives: "d7e3f1a"
💾 Database Design — Where Do We Store the Mappings?
Every URL shortener needs a database to store the relationship between short codes and long URLs. Think of it as a giant dictionary 📖:
DATABASE TABLE: url_mappings ┌────────────┬─────────────────────────────────────┬──────────────┬────────────┬──────────┐ │ short_code │ long_url │ created_at │ expires_at │ user_id │ ├────────────┼─────────────────────────────────────┼──────────────┼────────────┼──────────┤ │ 3xKp9 │ https://amazon.in/dp/B09G9HD6PD... │ 2025-01-01 │ 2026-01-01 │ user_42 │ │ mZ7qR2w │ https://docs.google.com/spreadsh... │ 2025-03-15 │ NULL │ user_17 │ │ q0U88 │ https://github.com/torvalds/linux │ 2025-06-20 │ NULL │ user_01 │ └────────────┴─────────────────────────────────────┴──────────────┴────────────┴──────────┘ INDEX on 'short_code' column — makes lookups LIGHTNING fast! ⚡
🗂️ SQL vs NoSQL — Which One to Use?
This is a very common question in System Design interviews! Here is a simple way to think about it:
✅ Use NoSQL (like Cassandra or DynamoDB) when:
- You expect billions of records (massive scale)
- Your data is simple key → value (short code → long URL)
- You need horizontal scaling across multiple machines
- Bitly, TinyURL, and most big players use this approach!
- You expect billions of records (massive scale)
- Your data is simple key → value (short code → long URL)
- You need horizontal scaling across multiple machines
- Bitly, TinyURL, and most big players use this approach!
⚠️ Use SQL (like PostgreSQL or MySQL) when:
- You are building a small to medium URL shortener
- You need complex queries (user reports, analytics joins)
- ACID transactions matter to you (strong data consistency)
- Great for startup projects and internal tools!
- You are building a small to medium URL shortener
- You need complex queries (user reports, analytics joins)
- ACID transactions matter to you (strong data consistency)
- Great for startup projects and internal tools!
⚡ Caching — Making Everything 100x Faster!
Here is a real-life analogy. Imagine your teacher asks you a maths question. You could go home, open a textbook, find the answer, and come back — that takes 10 minutes. OR you could remember the answer in your head from yesterday — that takes 1 second! 🧠
Caching is exactly this — storing the most popular answers in super-fast memory so you don't have to go to the slow database every time.
WITHOUT CACHE: WITH CACHE (Redis): ────────────── ─────────────────── User → Server → Database User → Server → Cache (⚡ 1ms) (takes ~50–100ms per lookup) (only miss goes to DB ~50ms) For 10,000 requests/second: Cache hit rate: ~90% = 10,000 database reads/second! = Only 1,000 DB reads/second = Database gets overwhelmed! 💥 = Database is happy!
📝 What does this code do?
The code below shows how to use Redis as a cache. When someone clicks a short URL, we first check Redis (super fast memory). If found there, we send the answer immediately. If not found, we go to the database and then save the result in Redis for next time.
The code below shows how to use Redis as a cache. When someone clicks a short URL, we first check Redis (super fast memory). If found there, we send the answer immediately. If not found, we go to the database and then save the result in Redis for next time.
# Python Example — Redis Caching with Cache-Aside Pattern
import redis
import psycopg2 # PostgreSQL database driver
# Connect to Redis cache (lives in super-fast memory)
cache = redis.Redis(host='localhost', port=6379, db=0)
# Connect to PostgreSQL database (lives on disk, slower)
db = psycopg2.connect("dbname=urlshortener user=admin")
def get_long_url(short_code):
"""
Tries to find the long URL for a given short code.
Step 1: Check cache first (lightning fast ⚡)
Step 2: If not in cache, check database
Step 3: Save result in cache for next time
"""
# Step 1: Check the cache first!
cached_url = cache.get(short_code)
if cached_url:
print("✅ Cache HIT! Returned instantly.")
return cached_url.decode('utf-8')
# Step 2: Cache miss — go to the database
print("❌ Cache MISS — querying database...")
cursor = db.cursor()
cursor.execute(
"SELECT long_url FROM url_mappings WHERE short_code = %s",
(short_code,)
)
result = cursor.fetchone()
if result:
long_url = result[0]
# Step 3: Save in cache for next time (expire after 1 hour)
cache.setex(short_code, 3600, long_url)
print("💾 Saved to cache for future requests.")
return long_url
return None # URL not found
# Test it!
print(get_long_url("3xKp9")) # First time: Cache MISS → DB
print(get_long_url("3xKp9")) # Second time: Cache HIT ⚡
↩️ HTTP Redirects — 301 vs 302 (The Hidden Superpower!)
When you click a short URL, the server sends back a special instruction to your browser: "Hey browser! Go to this other address instead!" This instruction is called an HTTP Redirect. There are two important types and they behave very differently!
HTTP 301 — PERMANENT Redirect HTTP 302 — TEMPORARY Redirect ───────────────────────────── ────────────────────────────── "This link has MOVED FOREVER." "This link is temporarily here." Browser behaviour: Browser behaviour: ✅ Remembers the redirect in cache ✅ Always asks the server first ✅ Never asks the server again ✅ Perfect for click tracking! ❌ Cannot track click counts! ✅ Server controls the destination Used for: Permanent URL changes Used for: URL shorteners with analytics SEO: Transfers page ranking ✅ SEO: No ranking transfer ❌ 👆 THIS IS WHY most URL shorteners use 302!
✅ Best Practice :
Use HTTP 302 for URL shorteners. This ensures every click passes through your server so you can count clicks, record geolocation, device type, and other analytics data before sending the user to the destination.
Use HTTP 302 for URL shorteners. This ensures every click passes through your server so you can count clicks, record geolocation, device type, and other analytics data before sending the user to the destination.
📊 Analytics — Tracking Every Click Like a Detective!
One of the most valuable features of a URL shortener is the ability to see who clicked your link, from where, and when. This is pure gold for marketers and businesses! 🕵️
ANALYTICS DATA CAPTURED FOR EVERY CLICK: ────────────────────────────────────────── ┌─────────────────────────────────────────────────┐ │ Short Code: 3xKp9 │ │ Timestamp: 2025-11-15 14:32:05 UTC │ │ IP Address: 103.24.xx.xx (masked for privacy)│ │ Country: India 🇮🇳 │ │ City: Mumbai │ │ Device: Mobile (iPhone 16) │ │ Browser: Safari 18.2 │ │ OS: iOS 18.1 │ │ Referrer: instagram.com │ └─────────────────────────────────────────────────┘
🏗️ Architecture: How Analytics Works at Scale
For a system handling millions of clicks per day, you cannot write analytics data to a database synchronously (i.e., right when the click happens). That would slow everything down! Instead, modern systems use an Async Message Queue:
Click Happens
│
▼
App Server redirects user IMMEDIATELY (fast! ⚡)
│
│ (at the same time, without waiting...)
▼
Publishes click event to Kafka/SQS Queue 📨
│
▼
Analytics Consumer Service picks up events
│
▼
Writes to Analytics Database (ClickHouse / BigQuery)
│
▼
Dashboard shows real-time charts 📊
💡 Why Use a Message Queue (Kafka)?
It's like a post box! Your server drops the click event in the post box and immediately moves on to serve the next user. A separate worker picks up the events from the post box at its own pace and processes them. No delays for the user! 📮
It's like a post box! Your server drops the click event in the post box and immediately moves on to serve the next user. A separate worker picks up the events from the post box at its own pace and processes them. No delays for the user! 📮
🚀 Scalability — Handling 10 Billion Clicks Per Month!
This is where System Design gets really interesting! How does a URL shortener stay fast and reliable even when millions of people click links at the same time?
Strategy 1: 🔄 Load Balancing
Without Load Balancer: With Load Balancer:
────────────────────── ────────────────────
All 10,000 users hit Load Balancer
ONE server → it crashes! 💥 ├── Server 1 (handles 2,500 users)
├── Server 2 (handles 2,500 users)
├── Server 3 (handles 2,500 users)
└── Server 4 (handles 2,500 users)
Everyone is happy!
Strategy 2: 🌍 CDN — Content Delivery Network
Imagine if Bitly's main servers are in New York. When someone in Mumbai clicks a link, the request travels from Mumbai → New York → Mumbai. That's really slow! 🐢
A CDN places servers all around the world. The Mumbai user gets served by a server in Mumbai — instant response! ⚡
Strategy 3: 📐 Database Sharding
SHARDING = Splitting the database across multiple machines Example: Splitting by first character of short code: Shard 1: Handles short codes starting with A–H (DB Server 1) Shard 2: Handles short codes starting with I–P (DB Server 2) Shard 3: Handles short codes starting with Q–Z (DB Server 3) Shard 4: Handles short codes starting with 0–9 (DB Server 4) → Each database server handles only 25% of the load! → You can add more shards as you grow! 📈
🔒 Security — Keeping Bad Actors Out!
URL shorteners are unfortunately a favourite tool for spammers and hackers — because the short URL hides the real destination. Here is how modern systems fight back:
🚫 DON'T Allow These!
❌ Shortening links to phishing websites
❌ Shortening malware download links
❌ Rate limits bypass using bots (creating 1 million short URLs in a second!)
❌ Shortening links to illegal or CSAM content
❌ Shortening links to phishing websites
❌ Shortening malware download links
❌ Rate limits bypass using bots (creating 1 million short URLs in a second!)
❌ Shortening links to illegal or CSAM content
✅ Security Best Practices :
✅ Scan every URL against Google Safe Browsing API before shortening
✅ Implement Rate Limiting — max 100 short URLs per user per day
✅ Add CAPTCHA for anonymous users
✅ Show a Preview Page before redirecting (user sees the destination first)
✅ Allow users to report malicious links
✅ Expire unused links after 90 days
✅ Log all IP addresses for abuse investigation
✅ Scan every URL against Google Safe Browsing API before shortening
✅ Implement Rate Limiting — max 100 short URLs per user per day
✅ Add CAPTCHA for anonymous users
✅ Show a Preview Page before redirecting (user sees the destination first)
✅ Allow users to report malicious links
✅ Expire unused links after 90 days
✅ Log all IP addresses for abuse investigation
✨ Vanity URLs — Custom Short Links (Premium Feature!)
Have you ever seen a link like https://bit.ly/IndiaVsAustralia?
That is a vanity URL — a custom, human-readable short code chosen
by the user instead of a random one. This is a popular premium feature!
REGULAR SHORT URL: https://bit.ly/3xKp9Wm VANITY SHORT URL: https://bit.ly/IndiaVsAustralia The second one is: ✅ Easier to remember ✅ More trustworthy (you know where it goes!) ✅ Great for branding (companies love this!) ✅ Perfect for printed materials, ads, and campaigns
📝 What does this code do?
The function below handles creating a vanity (custom) URL. It first checks if the custom name is already taken by someone else. If available, it reserves it for the user. If taken, it suggests an alternative!
The function below handles creating a vanity (custom) URL. It first checks if the custom name is already taken by someone else. If available, it reserves it for the user. If taken, it suggests an alternative!
# Python Example — Creating a Vanity URL
import re
RESERVED_WORDS = ['admin', 'api', 'login', 'dashboard', 'help', 'support']
def create_vanity_url(custom_code, long_url, db):
"""
Creates a custom (vanity) short URL chosen by the user.
Rules:
- Must be 3–30 characters long
- Only letters, numbers, and hyphens allowed
- Cannot be a reserved system word
- Must not already be taken
"""
# Rule 1: Check length
if not (3 <= len(custom_code) <= 30):
return {"error": "Custom code must be 3–30 characters long!"}
# Rule 2: Check for valid characters (letters, numbers, hyphens only)
if not re.match(r'^[a-zA-Z0-9-]+$', custom_code):
return {"error": "Only letters, numbers, and hyphens allowed!"}
# Rule 3: Block reserved system words
if custom_code.lower() in RESERVED_WORDS:
return {"error": f"'{custom_code}' is a reserved word. Please choose another!"}
# Rule 4: Check if already taken in the database
existing = db.get_by_short_code(custom_code)
if existing:
suggestion = custom_code + "-2025"
return {
"error": f"'{custom_code}' is already taken!",
"suggestion": suggestion
}
# All checks passed! Save to database.
db.save(short_code=custom_code, long_url=long_url, is_vanity=True)
return {"success": True, "short_url": f"https://bit.ly/{custom_code}"}
# Test it!
result = create_vanity_url("IndiaVsAustralia", "https://cricinfo.com/match/1234", db)
print(result) # {"success": True, "short_url": "https://bit.ly/IndiaVsAustralia"}
⏰ Link Expiry — Short URLs That Self-Destruct!
Sometimes you want a link that works only for a limited time. Imagine sending a discount code link that is only valid for 24 hours, or a file download link that expires after one use. This is called a Time-to-Live (TTL) or link expiry feature.
Use Cases for Expiring Links: ───────────────────────────── ⏱️ Password reset links → expire after 15 minutes 🎟️ Event ticket links → expire after the event date 💸 Flash sale discount links → expire after 24 hours 📄 Document sharing links → expire after 7 days 🔑 One-time-use download links → expire after first click Redis TTL makes this super easy! SET short_code long_url EX 86400 (EX = expire in seconds, 86400 = 1 day)
🏛️ The Complete System Design — Hero Level!
Now that you understand every individual component, let's put it all together into one master architecture diagram. This is what a production-grade URL shortener looks like:
┌──────────────────────────────────────────────────────────────────────────────┐
│ COMPLETE URL SHORTENER ARCHITECTURE │
└──────────────────────────────────────────────────────────────────────────────┘
USER / BROWSER
│
▼
┌─────────────────────────────────────────────────┐
│ CDN (Cloudflare / AWS CloudFront) │ ← Serves users from
│ Global Edge Servers in 200+ cities │ the nearest location
└────────────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────┐
│ Load Balancer (AWS ALB / Nginx) │ ← Distributes traffic
└────┬──────────────┬──────────────┬──────────────┘ evenly
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ App │ │ App │ │ App │ ← Multiple app servers
│ Server 1│ │ Server 2│ │ Server 3│ for high availability
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────┐
│ Redis Cache Cluster │ ← First stop for lookups
│ (Stores 1M most popular short codes) │ ~1ms response time
└────────────────────┬────────────────────────────┘
│ (Cache miss only)
▼
┌─────────────────────────────────────────────────┐
│ Primary Database (Cassandra / PostgreSQL) │ ← Permanent data store
│ Sharded across multiple nodes │ ~10-50ms response
└────────────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────┐
│ Analytics Pipeline │
│ Kafka Queue → Spark Streaming → ClickHouse │ ← Real-time analytics
└─────────────────────────────────────────────────┘
🔢 Back-of-the-Envelope Estimation (Interview Must-Know!)
System Design interviews always ask: "How much storage/bandwidth do you need?" Here is how the best engineers think about it — step by step:
ASSUMPTIONS:
─────────────────────────────────────────────
Daily new URLs created: 100 million/day
Daily URL redirects: 10 billion/day
Read-to-Write ratio: 100:1
Average long URL length: 200 characters = 200 bytes
Average short code length: 7 characters = 7 bytes
Retention period: 5 years
STORAGE CALCULATION:
─────────────────────────────────────────────
Total URLs in 5 years:
100 million/day × 365 × 5 = 182.5 billion records
Storage per record:
200 bytes (long URL)
+ 7 bytes (short code)
+ 50 bytes (metadata: user, timestamp, etc.)
= ~257 bytes per record
Total storage needed:
182.5 billion × 257 bytes ≈ 46.9 TB
(With 3x replication for redundancy: ~141 TB)
BANDWIDTH CALCULATION:
─────────────────────────────────────────────
Redirect requests per second:
10 billion/day ÷ 86,400 seconds ≈ 115,740 requests/second
Write requests per second:
100 million/day ÷ 86,400 seconds ≈ 1,157 requests/second
✅ Interview Tip!
Always start with assumptions, state them clearly, and then calculate. Interviewers care more about your reasoning process than the exact numbers. Being systematic and confident is what earns you the job offer! 🎯
Always start with assumptions, state them clearly, and then calculate. Interviewers care more about your reasoning process than the exact numbers. Being systematic and confident is what earns you the job offer! 🎯
🌐 Top URL Shortener Services
Let's take a quick look at the major players in the URL shortening market today and what makes each one special:
SERVICE BEST FOR SPECIAL FEATURES
────────── ───────────────────────── ──────────────────────────────────────
bit.ly Businesses & marketing Analytics dashboard, custom domains,
QR code generation, A/B testing links
TinyURL Simplicity No account needed, permanent links,
custom aliases (free!)
Rebrandly Brand-focused teams Custom domain names, team collaboration,
UTM parameter builder
Short.io Developers & APIs REST API, webhook support,
detailed click analytics, SSL
t.ly Simplicity + Speed Super fast, clean UI, free tier
Dub.co Open-source teams Fully open-source, self-hostable,
modern tech stack (Next.js + Redis)
💡 Trend: Open-Source URL Shorteners are Rising!
Many companies now prefer to self-host their own URL shortener using open-source tools like Dub.co or YOURLS. This gives them full control over data privacy, no vendor lock-in, and zero per-click fees. If you are building a SaaS product, this is worth exploring! 🔑
Many companies now prefer to self-host their own URL shortener using open-source tools like Dub.co or YOURLS. This gives them full control over data privacy, no vendor lock-in, and zero per-click fees. If you are building a SaaS product, this is worth exploring! 🔑
🛠️ Build Your Own URL Shortener — Step by Step!
Let's now build a minimal but working URL shortener using Python + Flask so you can see all the concepts come to life in real code!
📝 What does this full application do?
This is a complete mini URL shortener web application! It has two features:
1. POST /shorten — takes a long URL and returns a short code
2. GET /<short_code> — takes a short code and redirects to the long URL
It stores everything in a simple Python dictionary (in-memory database — for learning only!).
This is a complete mini URL shortener web application! It has two features:
1. POST /shorten — takes a long URL and returns a short code
2. GET /<short_code> — takes a short code and redirects to the long URL
It stores everything in a simple Python dictionary (in-memory database — for learning only!).
# app.py — A minimal URL Shortener in Python Flask
# Install: pip install flask
from flask import Flask, request, redirect, jsonify, abort
import string
import random
app = Flask(__name__)
# Our "database" (in-memory dictionary for learning purposes)
url_database = {}
BASE_URL = "http://localhost:5000/"
CHARS = string.ascii_letters + string.digits # A-Z, a-z, 0-9
def generate_code(length=7):
"""Pick 7 random characters to create a unique short code."""
return ''.join(random.choices(CHARS, k=length))
@app.route('/shorten', methods=['POST'])
def shorten_url():
"""
API Endpoint 1: Create a short URL
Accepts: { "long_url": "https://amazon.in/dp/very-long-path" }
Returns: { "short_url": "http://localhost:5000/aB3xKp9" }
"""
data = request.get_json()
long_url = data.get('long_url')
if not long_url:
return jsonify({"error": "Please provide a long_url!"}), 400
# Generate a unique short code
short_code = generate_code()
while short_code in url_database:
short_code = generate_code() # Regenerate if collision
# Save the mapping
url_database[short_code] = long_url
print(f"✅ Saved: {short_code} → {long_url}")
return jsonify({
"short_url": BASE_URL + short_code,
"short_code": short_code
})
@app.route('/<short_code>', methods=['GET'])
def redirect_to_long(short_code):
"""
API Endpoint 2: Redirect using a short code
When someone visits http://localhost:5000/aB3xKp9
they get redirected to the original long URL automatically!
"""
long_url = url_database.get(short_code)
if not long_url:
abort(404) # Return "Not Found" if code doesn't exist
print(f"🔀 Redirecting: {short_code} → {long_url}")
return redirect(long_url, code=302) # HTTP 302 Temporary Redirect
if __name__ == '__main__':
app.run(debug=True, port=5000)
HOW TO TEST IT:
Step 1: Run the app
$ python app.py
Step 2: Create a short URL (use curl or Postman)
$ curl -X POST http://localhost:5000/shorten \
-H "Content-Type: application/json" \
-d '{"long_url": "https://www.amazon.in/dp/B09G9HD6PD"}'
Response:
{
"short_url": "http://localhost:5000/aB3xKp9",
"short_code": "aB3xKp9"
}
Step 3: Visit the short URL in your browser
→ http://localhost:5000/aB3xKp9
→ You get automatically redirected to Amazon! ✅
🎯Cheat Sheet — URL Shortener
If you have a System Design interview coming up, here are the exact talking points that will impress your interviewer:
✅ Things to ALWAYS Mention:
1. Clarify Requirements: Read vs Write ratio? Analytics needed? Expiry?
2. Short Code Generation: Prefer Base62 of auto-increment ID
3. Database Choice: NoSQL (Cassandra) for scale, SQL for small systems
4. Caching: Redis with Cache-Aside pattern, 80/20 rule (20% links = 80% traffic)
5. Redirect Type: HTTP 302 for analytics, HTTP 301 for SEO
6. Load Balancing: Multiple app servers behind a load balancer
7. Sharding: Horizontal database sharding for billions of records
8. Analytics: Async processing via Kafka — never block the redirect!
9. Security: Google Safe Browsing API + rate limiting + CAPTCHA
10. Estimation: Storage, bandwidth, QPS — always back with numbers!
1. Clarify Requirements: Read vs Write ratio? Analytics needed? Expiry?
2. Short Code Generation: Prefer Base62 of auto-increment ID
3. Database Choice: NoSQL (Cassandra) for scale, SQL for small systems
4. Caching: Redis with Cache-Aside pattern, 80/20 rule (20% links = 80% traffic)
5. Redirect Type: HTTP 302 for analytics, HTTP 301 for SEO
6. Load Balancing: Multiple app servers behind a load balancer
7. Sharding: Horizontal database sharding for billions of records
8. Analytics: Async processing via Kafka — never block the redirect!
9. Security: Google Safe Browsing API + rate limiting + CAPTCHA
10. Estimation: Storage, bandwidth, QPS — always back with numbers!
🚫 Common Mistakes to AVOID:
❌ Using random UUIDs as short codes (too long — 36 characters!)
❌ Forgetting to handle hash collisions
❌ Writing analytics data synchronously (blocks the redirect!)
❌ Not adding an index on the short_code column in the database
❌ Forgetting about link expiry / TTL handling
❌ Not considering abuse prevention and security scanning
❌ Using random UUIDs as short codes (too long — 36 characters!)
❌ Forgetting to handle hash collisions
❌ Writing analytics data synchronously (blocks the redirect!)
❌ Not adding an index on the short_code column in the database
❌ Forgetting about link expiry / TTL handling
❌ Not considering abuse prevention and security scanning
🏆 Summary
Look at how far you have come! Let's recap everything you now know:
- 📦 What a URL Shortener is — converting long URLs into tiny ones, like a post office for web links
- 🔑 Short Code Generation — Random, Base62 Encoding, and Hash-Based approaches
- 💾 Database Design — SQL for small scale, NoSQL (Cassandra) for billions of records
- ⚡ Redis Caching — Cache-Aside pattern for lightning-fast redirects
- ↩️ HTTP 301 vs 302 — Why URL shorteners use 302 for analytics
- 📊 Analytics Pipeline — Async processing with Kafka for real-time insights
- 🚀 Scalability Techniques — Load balancing, CDN, database sharding
- 🔒 Security — Safe Browsing API, rate limiting, abuse prevention
- ✨ Vanity URLs — Custom short codes for branding
- ⏰ Link Expiry — TTL-based self-destructing links
- 🛠️ Building Your Own — A working Python + Flask URL shortener
- 🎯 Interview Prep — Exactly what to say (and not say!) in System Design interviews
💡 Bonus Trend — AI-Powered URL Shorteners!
The newest generation of URL shorteners now use AI to:
🤖 Auto-generate vanity codes based on the content of the target page
🧠 Predict click-through rates before you publish a campaign
🔍 Detect malicious URLs using LLM-based content analysis (not just blocklists!)
📝 Auto-tag and categorize links for better organisation
This is where the industry is heading — a fascinating space to watch!
The newest generation of URL shorteners now use AI to:
🤖 Auto-generate vanity codes based on the content of the target page
🧠 Predict click-through rates before you publish a campaign
🔍 Detect malicious URLs using LLM-based content analysis (not just blocklists!)
📝 Auto-tag and categorize links for better organisation
This is where the industry is heading — a fascinating space to watch!
Happy building.
Comments
Post a Comment