Identity Resolution at Scale: The Unglamorous Foundation of Personalization

TL;DR.

• Identity resolution at scale is the process of stitching together a single person's many identifiers — emails, device IDs, cookies, hashed phone numbers, anonymous session tokens — into one persistent profile. Without it, personalization is a series of disconnected hallucinations.
• The two strategies are deterministic matching (exact joins on verified identifiers like email or user_id) and probabilistic matching (statistical inference from device fingerprint, IP+UA, behavioral patterns). Production systems run both, with deterministic as the spine.
• The dominant patterns in 2026 are warehouse-native (Hightouch, Snowflake, Databricks-resident) and platform graphs (LiveRamp's RampID, Segment's deterministic-only profiles). Warehouse-native keeps PII in your account; platform graphs trade ownership for off-the-shelf reach.
• The two failure modes that quietly ruin personalization are over-merging (two people collapsed into one profile, e.g. shared family devices) and identity drift (the same person re-fragmented because deterministic keys rotated). Both produce worse personalization than no personalization.
• Privacy law (GDPR Art. 6, CCPA, the post-cookie web) has made probabilistic matching legally and technically harder. The companies that will win at scale are the ones who design their identity graph for revocation and minimization from day one.

Most teams think personalization fails because the model is bad. It usually fails because the identity is wrong. The recommendation engine returns a perfect suggestion for user_id=4711, but 4711 is actually three different humans, two browsers, and a stale cookie. Identity resolution is the load-bearing layer no one writes about, and it eats more engineering time than the model itself.

This post walks through what doing identity resolution at scale actually looks like in production — the deterministic vs. probabilistic trade-off, the warehouse-native vs. platform-graph architectures, the specific failure modes that destroy personalization quality, and the privacy constraints that will shape the next five years. If you're building anything that needs to recognize a person across sessions, devices, or channels, this is the foundation.

What identity resolution at scale actually is

Identity resolution is the process of linking the many identifiers that belong to one person — email addresses, hashed phones, device IDs, cookie IDs, anonymous session tokens, loyalty numbers, login IDs — into a single persistent profile, then keeping that profile coherent as new signals arrive. The result is an identity graph: a data structure where nodes are identifiers, edges are confidence-scored links, and a "person" is a connected component.

The reason it matters more at scale is that the failure surface grows superlinearly. With 10k users you can hand-tune a join. With 10M users across 21 connected devices per household (Hightouch), every error in the stitching propagates into every downstream model, recommendation, attribution number, and email send. Bad identity makes good models look bad.

In our framing, identity resolution at scale is the layer underneath everything we've written about — the reference architecture for real-time personalization, the recommendation engine vs personalization layer distinction, the path of a recommendation. None of those work if the person on the other end isn't who you think they are.

Identity stitching: the deterministic spine

Deterministic identity stitching is the simplest, most defensible approach. You declare a set of canonical identifiers — typically user_id, email, phone_e164, loyalty_id — and you merge two records if and only if they share a value on at least one of them. No probabilities. No model. Just exact joins on values you've verified.

This is the approach Segment built its CDP around: their model is 100% deterministic, tying identifiers to a persistent user_id and refusing to guess. It's also the spine of every serious warehouse-native identity graph — including the one Hightouch writes into your Snowflake instance.

The strength of deterministic identity resolution is precision. If two records share an email, they are almost certainly the same person (modulo shared mailboxes, which is its own problem). The weakness is reach: most of your traffic is anonymous. A user who hasn't logged in this session has no user_id. A user who clicks a marketing email before signing up has an email but no device_id. A user who shows up in an ad attribution pipeline has a mobile_ad_id (MAID) and nothing else. Deterministic-only graphs have huge dead zones.

The pattern we recommend, and the one most production systems converge on: deterministic for the spine, probabilistic for the gaps.

Probabilistic matching: where the math gets fuzzy

Probabilistic matching estimates the likelihood that two records refer to the same person from patterns: device fingerprint (browser + OS + fonts + canvas hash), IP + user-agent + time-of-day, behavioral signals (the same person tends to visit the same product pages in the same order), or partial PII matches (same first-name, same zip, similar email). The output is a confidence score, not a boolean.

Industry references like LiveRamp's RampID and mParticle's hybrid model frame this as "high-reach" matching — you get coverage on the anonymous traffic where deterministic methods have nothing to join on. The cost is precision: probabilistic graphs will sometimes merge two people, and sometimes split one person into two.

In practice the system pivots on a confidence threshold: above it, the match is acted on (added to the identity graph, used to merge profiles); below it, the match is logged but ignored. Where you set this threshold is a business decision more than a technical one. Billing and compliance systems want very high confidence (false merges are catastrophic). Email subject-line personalization wants moderate confidence (a wrong name in a subject line is recoverable). Ad audience expansion can run at low confidence (the cost of a wrong impression is small).

A working architecture for identity resolution at scale looks roughly like:

1. Ingest raw events with whatever identifiers are present, keyed by an event-time anonymous_id.
2. Deterministic pass: union-find over canonical identifiers. Produce a stable person_id per connected component.
3. Probabilistic pass: for unmatched anonymous_ids, run a matching model (gradient-boosted classifier on identifier-pair features, or a learned graph-embedding model) and add edges above a configurable threshold.
4. Materialize: write the identity graph and the resolved person_id back to the warehouse so downstream models can join against it.

Identity graph: architecture patterns at scale

There are three dominant patterns for doing identity resolution at scale in 2026. Each makes different trade-offs on ownership, latency, and reach.

Warehouse-native graphs. Tools like Hightouch run identity resolution as a SQL workload inside your warehouse — Snowflake, BigQuery, Databricks, Redshift. The graph is a table you own. The match logic is SQL or DBT you can read. Snowflake's writeup describes Hightouch writing the resolved graph directly into the customer's Snowflake instance — no data leaves. This is the pattern we see winning for product teams who already live in the warehouse.

Platform graphs. LiveRamp's RampID is the canonical example: you upload hashed PII, LiveRamp matches it against its own consumer graph, and you get back anonymized RampIDs that can be activated against ad platforms. The win is reach (LiveRamp sees identifiers you don't). The cost is that your PII has to leave your environment, and the graph is opaque.

CDP-internal graphs. Segment, mParticle, and other CDPs run an identity graph as part of their ingestion pipeline. The graph lives inside the CDP, gets queried at write time, and is exposed through the CDP's profile API. Easy to start. Hard to migrate off.

In every case, the scale challenge is the same: comparing millions of identifier pairs to find the few that should be merged is computationally heavy. Most production systems precompute blocking keys (zip code, hashed-email prefix) and only score candidate pairs within a block, exactly the way classical record-linkage systems have done for decades (Fellegi & Sunter, 1969 is still the canonical reference).

Failure modes: over-merging and identity drift

Two failure modes will silently destroy personalization quality, and both are hard to detect without active monitoring.

Over-merging is when two distinct people are collapsed into one profile. The classic case is a shared family device: a father's streaming account watched on a TV that's also used by his teenage son and his son's friends (Amperity calls this "identity conflict"). The merged profile now has a recommendation history that's a mash-up of incompatible tastes. The personalization quality of the resulting profile is often worse than the cold-start state — see our piece on the cold-start problem and day-zero personalization. Pure probabilistic systems are most vulnerable; the fix is to require deterministic confirmation before merging high-impact attributes.

Identity drift is the opposite: one person gets re-fragmented into multiple profiles because the deterministic keys you relied on rotated. Apple's MAID became zeroed-out under ATT. Cookies expire on a 7-day rolling window under ITP. Users get a new phone number. Emails get retired. Every rotation creates a new orphan profile, and the user's behavior history is now split across two person_ids. Without active reconciliation, the graph quietly degrades.

The mitigation for both is the same: monitor the graph as a system, not as a one-shot pipeline. Watch the distribution of identifiers-per-person over time. Watch the rate of new person_id creation vs. unique-visitor count. Alert when those diverge.

Privacy: the constraint that has gotten louder every year

The legal and technical landscape around identity has changed more in the past three years than in the previous ten. GDPR Article 6 requires a lawful basis for processing personal data. CCPA / CPRA give users a right to deletion and a right to opt out of "sale or sharing." Apple's App Tracking Transparency cut MAID availability dramatically. Safari and Firefox block third-party cookies entirely; Chrome has been narrowing the surface in stages.

For any team running identity resolution at scale, this has three practical consequences:

• Probabilistic matching is harder to justify legally. "We inferred this person from device fingerprint + IP" is a weaker consent story than "we matched on email they typed into our form."
• Deterministic identifiers are more valuable, and more expensive to acquire. First-party email captures, logged-in sessions, and authenticated loyalty members are the new gold.
• The graph has to be designed for revocation. When a user requests deletion, you need to be able to find every node in their connected component and tombstone it. If you can't enumerate the graph cleanly, you can't comply.

This is also why knowledge graphs are aging better than vector embeddings for identity — graphs are auditable and editable, embeddings are opaque and immutable in practice.

What to monitor for identity resolution at scale in production

A short list of metrics that catch identity problems before they show up in user complaints:

• Identifiers per person, p50 and p99. If p99 climbs from 5 to 50, you have an over-merging problem.
• New person_id rate vs. new unique-visitor rate. Should track tightly; divergence means identity drift.
• Deterministic match rate. Percentage of new events that join to an existing person_id on a verified identifier. A drop here is usually an upstream form-tracking break.
• Probabilistic-acceptance rate by threshold. If you change the threshold, what fraction of pending matches got accepted? You should be able to plot this.
• Time-to-resolution. From the moment a new identifier is seen to the moment it's joined to a profile. For real-time personalization you want this under <100 ms; for daily-batch CDP work you want this under 1 hour.

How ×marble fits in

×marble is the personalization knowledge graph that sits on top of resolved identity. We don't replace your identity resolution layer — we assume you have one (warehouse-native Hightouch, Segment Profiles, a homegrown graph, whatever works). What we do is take the resolved person_id and turn it into a multi-modal preference graph that the rest of your stack — recommendations, briefings, ad targeting — can query.

If you're starting from zero, our recommendation is: build the deterministic spine yourself in the warehouse, plug in a probabilistic layer when you need reach, and use ×marble for the personalization graph on top. If you'd rather not build any of this, we run hosted versions across our sub-products — Vivo for daily AI video briefings, Video for YouTube, Marble × Music for Spotify and Apple Music. The same identity model powers all three. Start at timesmarble.com.

FAQ

What is identity resolution at scale and why is it important?

Identity resolution at scale is the process of linking many identifiers — emails, device IDs, cookies, hashed phone numbers, anonymous session tokens — to a single persistent profile per person, across millions of users and continuously over time. It matters because every downstream system (recommendations, attribution, email, ads) assumes a coherent identity. When the identity is wrong, every personalization decision is wrong. Identity resolution at scale is the difference between personalization that feels uncanny-good and personalization that feels broken.

What is the difference between deterministic and probabilistic identity matching?

Deterministic matching joins records on exact, verified identifiers (email, user_id, phone_e164, loyalty_id). It is high-precision, low-reach. Probabilistic matching uses statistical models on signals like device fingerprint, IP+user-agent, and behavioral patterns to estimate the probability that two records refer to the same person. It is higher-reach but lower-precision. Production identity graphs almost always combine both, with deterministic as the spine.

What is an identity graph?

An identity graph is the data structure behind identity resolution: nodes are identifiers, edges are confidence-scored links, and a "person" is a connected component of the graph. The graph is queried by downstream systems to resolve any incoming identifier to a canonical person_id. The same graph is also the substrate for personalization layers like ×marble that need to reason over a person's full multi-channel history.

What is identity stitching?

Identity stitching is the operational side of identity resolution — the act of merging records that belong to the same person and splitting records that were wrongly merged. It runs continuously: every new event has the potential to add a new edge to the graph or invalidate an existing one. At scale, stitching is a streaming workload, not a batch job.

What are the biggest failure modes when running identity resolution at scale?

The two failure modes that destroy personalization quality are over-merging (two people collapsed into one profile, common with shared family devices) and identity drift (one person fragmented across multiple profiles because deterministic keys rotated, e.g. MAID zeroed-out, cookies expired, phone changed). Both produce worse outputs than no personalization at all, and both are invisible without active monitoring of identifiers-per-person and new-person_id rate.

Further reading

• Reference architecture for real-time personalization — where the identity graph sits in the broader stack.
• Cold-start problem and day-zero personalization — what to do before the identity is even resolved.
• Knowledge graphs vs. vector embeddings for personalization — why graphs win on auditability, which matters for privacy.
• The marketing engineer's personalization stack — where identity resolution fits into the broader tooling layout.
• LiveRamp on probabilistic vs. deterministic — the canonical industry framing from a platform-graph vendor.
• Snowflake on warehouse-native identity resolution — the architecture pattern we see winning for product teams.
• Record linkage (Wikipedia) — the academic foundation, including Fellegi-Sunter, that all of this is built on.