Skip to main content

Truecaller System Design: How Caller ID and Spam Detection Work at Scale

Calculating read time…

A number that has never texted you, never been saved, never been seen by your phone before — rings, and a name appears on your screen before the second ring.

That's a real-time reverse-lookup problem running against a database of billions of phone numbers, and the answer has to arrive in the few hundred milliseconds between "incoming call event fired" and "phone actually rings." Get it wrong and either the call rings with no answer (bad UX) or, worse, the wrong name shows up (trust destroyed instantly).

Truecaller solves this with the same category of engineering problem Twitter solves with fan-out — one write (a contact getting saved) potentially needs to be visible to millions of future readers, and the expensive work has to happen before the read, not during it. Let's design the whole system, the way you'd whiteboard it in a Google or Meta system design interview.

💡 Numbers That Matter — Interview-Scale Assumptions

📱 Assume ~400M+ monthly active users voluntarily uploading contacts

🗂️ Assume the reverse index holds billions of unique phone numbers, each with multiple candidate names

⚡ Ring-time lookup latency budget: under 200ms — has to beat the ringtone

✍️ Write volume: contact-sync events dwarf lookup events by orders of magnitude — this is a write-heavy ingestion, read-heavy serving system

The unique constraint: unlike Twitter, the "content" here is contested — thousands of different users can submit conflicting answers for the exact same key.

📑 In This Post


🏛️ Section 1: Mental Model — A Shared Phonebook + Postal System

Before services and databases, build the right picture in your head.

💡 The Shared Notebook Analogy

Every phone in a school has its own contact notebook. Now imagine a librarian collects a photocopy of every notebook and glues them into one giant reference book, organized by phone number instead of by name. When a stranger calls you, you don't search your own notebook — you search the giant glued-together one. If even one kid in the whole school saved that number, you find it.

🏛️ Analogy📞 Truecaller Reality🔧 Engineering Concept
A kid writes a name next to a numberA user saves a contact and syncs itWrite Path / Ingestion
Librarian glues every notebook into one book, organized by numberA batch job builds a global reverse index, keyed by numberPrecomputed Reverse Index
5 kids wrote 5 different names for the same numberThousands of conflicting saved names for one numberConflict Resolution / Voting
You flip straight to the right page instantlyRing-time caller ID lookupRead Path / Cache Lookup
A kid rips a page out and asks the librarian to remove their entryA user requests unlisting / opt-outConsent & Data Deletion Pipeline

🗺️ Section 2: The Big Picture — Full Architecture

Here's the bird's-eye view. We deep-dive into every box below.

📱 Client Apps (Mobile / OS-level integration) → ⚖️ Load Balancer → 🚪 API Gateway (auth + rate limit)
⬇️
📇 Contact Sync Service  |  🔎 Lookup Service  |  🚫 Spam Report Service  |  🏢 Business Verification Service
⬇️
📨 Kafka Event Spine — topics: contact-uploads, spam-reports, opt-outs
⬇️
🧮 Resolution Engine (Spark/batch) → writes → 🗺️ Sharded KV Store (number → resolved name + spam score)
⬇️
🔴 Redis Cache (hot numbers) ⟷ served to 📱 clients via the Lookup Service at ring-time

🔐 Section 3: Privacy-Safe Number Identity — Normalization & Hashing

Before you can store or match a phone number at all, you need one consistent identity for it. "9876543210", "+91 98765 43210", and "098765-43210" must all resolve to the exact same key — this is the equivalent of Twitter's Snowflake ID problem, except here the ID isn't generated, it's derived from the number itself.

💡 The Locker-Key Analogy

Think of a gym locker system where your locker number is generated by running your membership ID through a fixed formula. Anyone can compute the same locker number from the same ID, but nobody can reverse the locker number back into your original ID. That one-way formula is a cryptographic hash.

Step 1 — Normalize: every number is converted to E.164 format (+<country code><national number>) so formatting differences collapse into one canonical string.

Step 2 — Hash: the normalized number is passed through SHA-256 with a server-side salt before it's used as a sharding key. This means the sharding/routing layer never needs to hold the plaintext number in memory unnecessarily, and it gives you a uniformly distributed key for consistent hashing.

Step 3 — Shard: the hash prefix determines which shard of the KV store owns that number — the same consistent-hashing technique used to distribute keys evenly across nodes in any large distributed database (Cassandra, DynamoDB).

✅ Interview signal: naming "consistent hashing for shard routing" and "normalize before you hash, hash before you shard" in the same breath is exactly the kind of detail that separates a strong system design answer from a shallow one.

📤 Section 4: The Write Path — What Happens When You Sync Contacts

1  Consent screen → permission granted. The app requests contacts permission with an explicit, revocable consent flow.

⬇️

2  Batch upload over TLS. (number, saved-name) pairs are normalized on-device, then uploaded in a compressed batch — not one request per contact.

⬇️

3  API Gateway validates + rate limits. Checks auth token and enforces an upload-rate cap per account (token bucket, same primitive Twitter uses for tweet limits).

⬇️

4  Contact Sync Service hashes + publishes to Kafka. Each pair is normalized, hashed, and emitted as an event on the contact-uploads topic — the write itself is not synchronous with resolution.

⬇️

5  Resolution Engine consumes asynchronously. A streaming/batch consumer increments the vote counters for that number — this is where Section 5's hard problem lives.

🌊 Section 5: The Hard Problem — Name Resolution at Conflicting Scale

This is Truecaller's version of Twitter's fan-out problem — the single hardest, most interview-worthy piece of this system. The challenge isn't storage, it's arbitration: a number can have thousands of conflicting candidate names, and the system has to pick one winner, fast, resistant to abuse, at global scale.

📝 Approach 1: Resolve on Read

✅ Always reflects the very latest votes    ✅ No wasted precompute for numbers nobody looks up
❌ Every ring-time lookup would need to scan and re-rank potentially thousands of votes    ❌ Blows the 200ms latency budget instantly at scale

📖 Approach 2: Resolve on Write (Precompute Everything)

✅ Ring-time lookup becomes a trivial O(1) read    ✅ Predictable, low latency regardless of vote count
❌ Recomputing on every single incoming vote is wasteful for low-traffic numbers    ❌ Naively synchronous recompute would create a write storm identical to Twitter's celebrity fan-out problem

🏆 The Hybrid Solution

Batch resolution (majority of numbers): a periodic Spark job re-ranks votes for numbers that changed since the last run, using weighted voting — not a simple count. Each vote is weighted by voter-account trust score (age, diversity of network, past reporting accuracy), so 5,000 low-trust accounts voting in the same hour can't outvote 300 long-standing users.

Streaming fast-path (hot numbers): numbers crossing a lookup-frequency threshold get incremental, near-real-time recomputation via a streaming consumer — the same "hot key" special-casing Twitter applies to celebrity accounts, just triggered by lookup volume instead of follower count.

Stability guard: a resolved name only flips once a new candidate's weighted score exceeds the incumbent's by a margin, and holds for a minimum window — preventing a single well-timed manipulation burst from instantly overwriting an established identity.

⚠️ Interview tip: when the interviewer pushes on "what if the resolved name is wrong or stale," your answer is the stability guard + margin threshold — not "add more servers." Conflict resolution under adversarial input is the actual system design skill being tested here.

🏠 Section 6: The Read Path — Assembling a Caller ID in Under 200ms

1  Incoming call event fires with the raw number → normalized on-device instantly.

⬇️

2  Local device contacts checked first — if it's already saved, zero network calls needed.

⬇️

3  If unsaved, the hashed number is sent to the Lookup Service, which checks Redis first.

⬇️

4  Cache miss falls through to the sharded KV store, routed via the hash prefix from Section 3.

⬇️

5  Resolved name + spam score returned; result is written back into Redis to warm the cache for the next lookup.

✅ Why this hits the latency budget: every expensive decision (which name won, is this spam) was already made in Section 5, offline. The read path never re-ranks votes — it's always a cache/KV read, which is the entire reason the 200ms budget is achievable at all.

📈 Section 7: Spam Detection — Velocity, Not Just Volume

A number that's always gotten a handful of spam reports isn't urgent. A number that got 50 spam reports in the last 10 minutes is a live telemarketing or fraud campaign — and needs to propagate fast.

NumberReports/hour trendVerdict
A — steady low-level reports12 → 14 → 11Not flagged ❌
B — sudden burst8 → 20 → 640Flagged as active spam campaign ✅

Reports flow through Kafka into a sliding-window counter (5-minute buckets, checked against the trailing hour). A spike that crosses both an absolute threshold and a relative-growth threshold triggers an immediate fast-path recompute — the same hot-key special-casing from Section 5 — bypassing the normal batch cycle so the label updates within minutes, not the next batch run.

🗄️ Section 8: The Data Layer — Databases & Why Each Was Chosen

🗺️ Sharded Wide-Column Store (Cassandra / HBase-style): the source of truth for number → vote-list and number → resolved-record. Sharded by the hash prefix from Section 3; chosen because writes (votes) vastly outnumber the need for complex joins.

🔴 Redis: hot-path cache for the most recently/frequently looked-up numbers, plus rate-limit token buckets. This is what makes the 200ms budget realistic.

📨 Kafka: the event spine connecting contact-sync, spam-report, resolution, and opt-out pipelines — decoupling ingestion speed from resolution speed.

🔍 Search Index (Elasticsearch-style): powers business/number search and admin tooling — separate from the ring-time hot path so heavy search queries never compete with latency-critical lookups.

🐘 Data Lake (HDFS/S3 + Spark): cold storage for historical votes and reports, used to train spam-classification models and to run the batch Resolution Engine from Section 5.

🛡️ Section 9: Privacy & Compliance Architecture

This layer is what makes the whole design defensible, and it's a section interviewers at tier-1 companies specifically probe for once they hear "crowdsourced personal data."

Data minimization: only (number, name) pairs are uploaded — not full contact metadata, photos, or message history.

Consent as a first-class object: permission state is stored per-user, per-scope, and is revocable — revoking triggers a deletion event on the same Kafka spine used for ingestion.

Right to opt-out / unlisting: a dedicated pipeline lets any number's owner request removal; this writes a tombstone record that suppresses that number from resolution regardless of remaining votes.

Regional data residency: under regulations like GDPR and India's DPDP Act, sharding strategy may need to account for keeping certain users' data within specific geographic regions — a real constraint on the "just shard by hash" design from Section 3.

💻 Section 10: Code — Core Services in Pseudocode

📌 What This Code Does (Read Before The Code!)

This is the Contact Sync Service — the entry point of the entire write path from Section 4. When your phone uploads a batch of contacts, this is the first piece of backend code that touches it.

First, it checks the rate limiter — if this account is uploading too fast (a sign of abuse or a buggy client), the request is rejected before any real work happens. This protects the Resolution Engine downstream from being flooded.

Then, for every (number, name) pair in the batch, it does the two things from Section 3: normalizes the number into E.164 format, and hashes it with a server-side salt. This is deliberate — from this point onward, no downstream service ever needs to see the raw, human-readable phone number to do its job.

Finally, it doesn't write to any database itself. It only publishes an event to Kafka and returns. This is the most important design decision in this snippet: the Contact Sync Service's job ends the moment the event is published — deciding whose name "wins" is deliberately not its problem. That separation is what keeps the sync response fast and is exactly the async, decoupled write pattern the whole system depends on.

def sync_contacts(user_id, contact_batch):
    if not rate_limiter.consume(user_id, action="contact_sync"):
        return error(status=429, msg="Upload rate limit exceeded")

    for raw_number, saved_name in contact_batch:
        number = normalize_e164(raw_number)
        number_hash = sha256(number + SERVER_SALT)

        kafka.publish(topic="contact-uploads", message={
            "number_hash": number_hash,
            "name":        saved_name,
            "voter_id":    user_id,
            "trust_score": trust_service.get_score(user_id),
            "timestamp":   now()
        })

    return success({"synced": len(contact_batch)})

📌 What This Code Does (Read Before The Code!)

This is the Resolution Engine — the piece of code doing the actual hard work described in Section 5. It's the direct equivalent of Twitter's fan-out worker, except instead of pushing a tweet ID to millions of timelines, it's arbitrating between thousands of conflicting votes for a single number.

It only runs on changed_number_hashes — numbers that actually received new votes since the last run. This is a deliberate efficiency choice: there's no reason to re-rank a number nobody has voted on recently.

For each number, it pulls every vote ever cast, but instead of counting them 1-for-1, it sums each voter's trust_score — this is the weighted-voting mechanism from Section 5 that stops a burst of low-trust or bot accounts from outvoting a smaller number of long-standing, trustworthy users.

The critical line is the MIN_MARGIN check. The system doesn't just take whichever name currently has the highest score — it only lets a new name overwrite the existing resolved name if it beats the runner-up by a meaningful margin. This is the "stability guard" from Section 5: it's what stops a single well-timed manipulation attempt from instantly flipping an established, trusted name.

Notice what actually gets written to the fast-read KV store: not the entire vote list (that stays in the slower vote store), just the single winning {name, spam_score, updated_at} record. That's what makes the read path in the next snippet a trivial lookup instead of a re-computation.

MIN_MARGIN = 0.15  # winning score must beat runner-up by this margin to flip

def resolve_batch(changed_number_hashes):
    for number_hash in changed_number_hashes:
        votes = vote_store.get_votes(number_hash)  # {name: [ (trust_score, ts), ... ]}

        scores = {
            name: sum(trust for trust, ts in v)
            for name, v in votes.items()
        }

        ranked = sorted(scores.items(), key=lambda kv: kv[1], reverse=True)
        current = resolved_index.get(number_hash)

        top_name, top_score = ranked[0]
        runner_up_score = ranked[1][1] if len(ranked) > 1 else 0

        if current is None or (top_score - runner_up_score) >= MIN_MARGIN:
            resolved_index.put(number_hash, {
                "name": top_name,
                "spam_score": spam_service.score(number_hash),
                "updated_at": now()
            })
        # else: keep current name — not enough margin to flip yet

📌 What This Code Does (Read Before The Code!)

This is the Lookup Service — the only piece of code in this entire design that runs while your phone is actually ringing, so every line in it is on the 200ms latency budget from the very top of this post.

The first thing it does is normalize and hash the incoming number, exactly the same way the Contact Sync Service did in the first snippet. Using the identical hashing logic on both the write side and the read side is what guarantees a number saved as "9876543210" on one phone matches a call arriving as "+91 98765 43210" on another — they collapse to the same key.

It checks Redis first. Notice there's no ranking, no vote counting, no scoring logic anywhere in this function — that's the whole point. All of that expensive work already happened offline in the Resolution Engine, so a cache hit here is just a dictionary read that takes microseconds.

Only on a cache miss does it fall through to the slower sharded KV store — and even then, it immediately writes the result back into Redis with a TTL before returning. This "cache-warm on miss" pattern means the very next person who calls that same number gets an instant cache hit instead of paying the KV-store cost again.

def lookup(raw_number):
    number_hash = sha256(normalize_e164(raw_number) + SERVER_SALT)

    cached = redis.get(f"caller_id:{number_hash}")
    if cached:
        return cached

    record = resolved_index.get(number_hash)
    if record is None:
        return {"name": None, "spam_score": 0}

    redis.set(f"caller_id:{number_hash}", record, ttl=3600)
    return record

✅ Notice the pattern: writes (votes), resolution (ranking), and reads (lookup) are three fully decoupled paths connected only through Kafka and the KV store. This separation — not any single clever algorithm — is the actual answer to "how does Truecaller scale."

🛡️ Section 11: Reliability

KV store replication: each shard has replicas; a leader failure promotes a replica automatically, so reads stay available during node failure.

Kafka durability: votes and spam reports are never lost on a consumer crash — the Resolution Engine resumes from its last committed offset.

Graceful degradation: if the Resolution Engine falls behind, ring-time lookups keep serving the last-known resolved name from cache — stale is far better than unavailable.

Idempotency: duplicate vote events (from retries) are deduplicated by a composite key of (voter_id, number_hash) so the same user can't accidentally double-vote.

🚦 Section 12: Rate Limiting & Abuse Protection

ActionLimitWhy
Contact sync batchCapped per account/dayPrevent bulk vote injection
Spam reportsCapped per account/hourPrevent coordinated false-flagging
Ring-time lookupsHigh ceiling, per-deviceLegit usage is naturally low-frequency

All limits are enforced with a token-bucket counter in Redis — the same primitive used for the hot lookup cache, so no extra infrastructure is needed. New/low-trust accounts get tighter buckets than established ones, tying directly back into the weighted-voting trust score from Section 5.

📈 Section 13: Scalability

Consistent hashing for shard routing — adding shards only reshuffles a small fraction of keys, not the whole dataset.

Async ingestion — contact syncs never block on resolution; Kafka absorbs traffic spikes and the Resolution Engine catches up at its own pace.

Read/write path separation — the expensive weighted-voting computation happens off the critical path, so read latency stays flat regardless of how contested a number is.

Stateless services — Lookup, Contact Sync, and Spam Report services hold no local state, so horizontal scaling during traffic spikes is just adding instances behind the load balancer.

🗺️ Section 14: Complete Architecture Map

📱 Client → ⚖️ Load Balancer → 🚪 API Gateway (auth + rate limit)
⬇️
📇 Contact Sync  |  🔎 Lookup  |  🚫 Spam Report  |  🏢 Business Verification
⬇️
📨 Kafka: contact-uploads · spam-reports · opt-outs
⬇️
🧮 Resolution Engine (Spark batch + streaming fast-path) → 🗺️ Sharded KV Store
⬇️
🔴 Redis Cache ⟷ served at ring-time  |  🔍 Search Index  |  🐘 Data Lake (model training)

📐 Section 15: Core Design Principles

🌊 Optimize for the read path. Lookups vastly outnumber contested-name conflicts at any given moment; doing the expensive ranking work at write/resolution time keeps the hot path a simple cache read.

📦 Async everything non-critical. Resolution, spam scoring, and search indexing never block the user-facing sync or lookup response.

🔑 Derive identity, don't generate it. Unlike Twitter's Snowflake IDs, the "key" here (the number) already exists — the engineering problem is normalizing and hashing it consistently, not generating uniqueness.

🎭 Different data, different arbitration. Regular numbers use scheduled batch resolution; hot/high-report numbers get a streaming fast-path — a direct parallel to Twitter's regular-user vs celebrity fan-out split.

🎓 Section 16: Interview Cheat Sheet

If asked to "Design Truecaller" (or any reverse-lookup/crowdsourced-identity system), here's your 5-step framework:

Step 1 — Clarify Requirements (5 min): Reverse lookup for unsaved numbers? Spam detection? Business verification? Opt-out/unlisting? Real-time or eventually-consistent naming acceptable?

Step 2 — Estimate Scale (5 min): Hundreds of millions of users, billions of stored numbers, write-heavy ingestion vs read-heavy serving, sub-200ms lookup latency budget.

Step 3 — High-Level Design (10 min): Write path (sync → Kafka → Resolution Engine → KV store) and read path (client → cache → KV store fallback) as two decoupled flows.

Step 4 — Deep Dive the Hard Problem (15 min): Conflicting-vote resolution — explain resolve-on-read vs resolve-on-write trade-offs, then the hybrid batch + streaming-fast-path solution with weighted trust scoring and a stability margin to resist manipulation.

Step 5 — Edge Cases (5 min): Number reassigned to a new owner (stale name until new votes cross the margin), coordinated fake-vote attack (trust-weighted scoring + rate limits), opt-out request (tombstone overrides all votes), hot number suddenly spamming (streaming fast-path bypasses batch delay).

🎉 Final Summary

🔐 Normalize then hash — E.164 normalization plus salted SHA-256 gives every number one consistent, privacy-safe identity for sharding.

✍️ Write path is fully async — contact syncs publish to Kafka and never block on resolution.

🌊 Name resolution is the real hard problem — weighted voting plus a stability margin resolves conflicting votes without being trivially gameable.

🏆 Hybrid batch + streaming fast-path — regular numbers resolve on a schedule; hot/high-report numbers get near-real-time recomputation.

🏠 Read path is a pure cache read — all the expensive ranking already happened offline, so ring-time lookup just reads Redis then falls back to the sharded KV store.

📈 Spam detection is velocity-based — a sudden spike in reports matters more than steady low-level volume.

🛡️ Privacy is architecture, not an afterthought — consent state, opt-out tombstones, and data minimization are first-class objects in the design, not bolt-ons.

✅ The Core Lesson: whenever many independent writers can produce conflicting answers for the same key, the system design question is never "how do I store this" — it's "how do I arbitrate this, fast, and resistant to manipulation, without making every reader pay the arbitration cost." That single idea is what an interviewer is actually listening for.

Happy Building! Keep designing for the read path. 🔥

Comments