From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents

Abstract: Anonymization is widely treated as a practical safeguard because re-identifying anonymous records was historically costly, requiring domain expertise, tailored algorithms, and manual corroboration. We study a growing privacy risk that may weaken this barrier: LLM-based agents can autonomously reconstruct real-world identities from scattered, individually non-identifying cues. By combining these sparse cues with public information, agents resolve identities without bespoke engineering. We formalize this threat as \emph{inference-driven linkage} and systematically evaluate it across three settings: classical linkage scenarios (Netflix and AOL), \emph{InferLink} (a controlled benchmark varying task intent, shared cues, and attacker knowledge), and modern text-rich artifacts. Without task-specific heuristics, agents successfully execute both fixed-pool matching and open-ended identity resolution. In the Netflix Prize setting, an agent reconstructs 79.2\% of identities, significantly outperforming a 56.0\% classical baseline. Furthermore, linkage emerges not only under explicit adversarial prompts but also as a byproduct of benign cross-source analysis in \emph{InferLink} and unstructured research narratives. These findings establish that identity inference -- not merely explicit information disclosure -- must be treated as a first-class privacy risk; evaluations must measure what identities an agent can infer.

• The paper demonstrates that LLM agents can reconstruct identities from weak cues using inference-driven linkage, achieving high success rates.
• It evaluates performance on legacy and modern datasets, showing LLM models outperform traditional de-anonymization methods with up to 80-100% linkage success.
• The study introduces the InferLink benchmark, revealing unintended privacy risks and emphasizing the need for robust, utility-preserving safeguards in LLM deployments.

Inference-Driven De-Anonymization in LLM Agents: Privacy Risks from Weak Cues

Introduction

The study "From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents" (2603.18382) provides a comprehensive analysis of how LLM-based agents can perform de-anonymization through inference-driven linkage, even when only fragmented, individually non-identifying cues are present in input data. The authors formalize a failure mode—inference-driven linkage, where agents reconstruct identities from weak cues, marking a substantial departure from prior privacy threats focused solely on the direct extraction or leakage of explicit identifiers.

Figure 1: Overview of Inference-Driven Linkage—an LLM agent reconstructs a specific identity $\hat{\imath}$ from fragmented information by combining weak cues from anonymized artifacts and corroborating auxiliary context.

Problem Definition and Threat Model

The central construct is an evaluation interface $$\Pi: (D_{\text{anon}}, D_{\text{aux}}) \mapsto (\hat{\imath}, \mathcal{E})$$, where an agent takes anonymized artifacts $D_{\text{anon}}$ and auxiliary context $D_{\text{aux}}$ as input and produces an identity hypothesis $\hat{\imath}$ with supporting evidence $\mathcal{E}$. Unlike classical membership inference or attribute prediction, this setup captures identity-level inference, where no single feature is identifying in isolation, but their aggregate enables linkage.

Key variations considered include:

• Task Framing (Intent): Benign analytic tasks versus explicit re-identification requests
• Fingerprint Type: Intrinsic, coordinate, and hybrid features as quasi-identifiers
• Attacker Knowledge: Zero-knowledge (ZK) or membership-knowledge (MK) scenarios

Evaluation on Classical Linkage Cases

The authors revisit the Netflix Prize [narayanan2008robust] and AOL search log incidents [aol_search_log] to compare the effectiveness of LLM-based agents against hand-engineered deanonymization pipelines.

In the Netflix scenario, the agent receives a pool of anonymized user records and a noisy auxiliary fragment, mirroring canonical de-anonymization conditions.

Figure 2: Netflix linkage success rate—LLM agent (GPT-5) outperforms the classical baseline, particularly under high noise and sparse fragment conditions.

Results:

• GPT-5 achieved a linkage success rate (LSR) of 79.2% (with only 2 auxiliary events, high perturbation), drastically surpassing the classical baseline at 56.0%.
• With higher information density (more auxiliary cues), agent performance converges to the baseline near-perfect regime.

In the AOL case, the methodology shifts to open-ended corroboration: agents analyze search logs, extract behavioral cues, construct candidate identities by web retrieval, and triangulate results with public records. Agents independently reconstructed and corroborated specific identities in 10 out of 40 privacy-sensitive histories, leveraging cross-source reasoning—such as linking query sequences to business registries, lifestyle triangulation, and tracing creative outputs.

InferLink: A Controlled Benchmark

Recognizing the limitations of historical cases (static overlap and lack of systematic variation), the authors introduce InferLink: a synthetic, controlled benchmark enabling precise variation of fingerprint structure, task framing, and attacker knowledge.

Figure 3: End-to-end pipeline of InferLink illustrating benchmark construction, scenario generation, paired-dataset synthesis, multi-turn interaction, and evaluation.

InferLink seeds each instance with a formal tuple $(f, \iota, \kappa)$, systematically varying conditions while enforcing unique ground-truth linkages. The data synthesis ensures that identity inference is non-trivial—no direct identifiers, only quasi-identifiers, and side-only features.

• Implicit: Benign analytic framing, no explicit linkage request
• Explicit-ZK: Explicit linkage request without specifying the target
• Explicit-MK: Explicit linkage with a named target

Key findings:

• Silent linkage occurs frequently. Claude 4.5 exhibited Linkage Success Rate (LSR) up to 80% in “Implicit” (benign task) cases, indicating de-anonymization occurs as a byproduct of utility-driven reasoning.
• Explicit prompts further increase risk: Claude 4.5 achieves up to 100% LSR under Explicit-MK; GPT-5 is also highly susceptible, especially for intrinsic/hybrid fingerprints.
• Prompt-based privacy defenses (system-level guardrails) can reduce LSR to near zero, but induce measurable utility costs, especially in models exhibiting over-refusal.

Case Studies on Modern Digital Traces

The study extends to contemporary, text-rich artifacts: (1) redacted professional interviews (Anthropic Interviewer data [handa2025interviewer]) and (2) anonymized ChatGPT logs.

• In the Anthropic Interviewer case, the LLM agent leverages technical/procedural narratives, methodological cues, and idiosyncratic phrasing to generate targeted search queries, eventually collapsing anonymity sets and attributing interviews to specific researchers. Six identity recoveries were confirmed, often via conjunctions of rare research workflows and role statements.

Figure 4: An Anthropic Interview qualitative example; the agent forms an academic persona profile from text, retrieves corroborating public evidence, and attributes interview content to a specific researcher.

• In ChatGPT logs, a single confirmed de-anonymization was achieved by progressive narrowing, integrating role, temporal, and topical signals to reduce candidate pools in an anonymity-set reduction process reminiscent of practical $k$-anonymity attacks.

In both cases, the identifying signal emerges from distributed weak cues—without explicit identifiers—demonstrating that current anonymization practices do not substantially mitigate identity inference risk in modern agentic environments.

Implications and Future Directions

This work establishes that LLM-based agents are potent deanonymizers, with capabilities that match or exceed domain-specific algorithms without any bespoke engineering. The results reveal that identity inference can occur even in benign workflows and under plausible prompt constraints—posing a privacy risk not directly captured by existing frameworks focused on explicit disclosure or direct identifier access.

Strong Claims:

• Modern LLM agents can match or surpass classical de-anonymization on legacy datasets.
• Silent, unintended identity linkage (LSR up to 80%) can emerge in scenarios absent explicit re-identification prompts.
• Prompt-level guardrails can mitigate, but also degrade, agent utility.

Contradictory/Noteworthy Aspects:

• Privacy failures arise from cross-source reasoning and evidence aggregation, not just direct leakage.
• Defenses that target explicit identifier handling are insufficient against inference-driven attacks.

Practically, the research implicates all deployments operating on heterogeneous or composite data where side information may exist. Theoretically, the study prompts reevaluation of privacy-warranting assumptions in agent design—particularly the residual risk from non-identifying, behavioral, or contextual data.

Future work directions include scaling systematic evaluations to larger ambiguity structures, integrating adversarial and defense-aware training, refining definitions of utility and harm, and developing alignment techniques capable of distinguishing legitimate analytical reasoning from privacy-violating inference.

Conclusion

The manuscript demonstrates that privacy risk from inference-driven linkage is not only plausible but prevalent in LLM-agent systems, both in structured benchmarks and real-world digital traces. Privacy risk assessments must be expanded to include identity inference—measuring what can be reconstructed, not only what is disclosed. As agentic deployments proliferate, comprehensive mitigation will necessitate advances in alignment, detection of dangerous cross-source inference, and robust, utility-preserving safeguards against deanonymization.

(2603.18382)