Attack Atlas: Sequence & GenAI Threat Analysis

• Attack Atlas is a dual-framework uniting sequence-based forensic investigation of system attacks with a hierarchical taxonomy for adversarial prompt attacks in GenAI.
• ATLAS/ATLASv2 utilizes high-fidelity telemetry logs and sequence models to achieve robust detection metrics (Precision ~0.88–0.90, F1 ~0.86–0.87) in multi-stage attack scenarios.
• Its GenAI taxonomy categorizes diverse prompt attack techniques, guiding red-teaming and blue-teaming methodologies for improved adversarial defense.

The term "Attack Atlas" encompasses two distinct but seminal contributions within security research: (1) a dataset-driven framework for sequence-based attack investigation in traditional system environments, known as ATLAS/ATLASv2, and (2) a hierarchical taxonomy and methodology for probing and defending against single-turn prompt attacks in generative AI (GenAI), especially LLMs. Each instantiation of the Atlas concept formalizes the empirical and methodological basis for evaluating adversarial behavior—whether at the level of forensic system telemetry or adversarial prompt engineering—providing practitioners and researchers with unifying frameworks for attack surface exploration, defense evaluation, and taxonomy-guided threat modeling (Riddle et al., 2023, Rawat et al., 2024).

1. Conceptual Overview

The "Attack Atlas" concept bifurcates into two principal research threads. In the first thread, originating with "ATLAS: A Sequence-based Learning Approach for Attack Investigation" and extended by ATLASv2, the Atlas comprises a labeled corpus of multistream system telemetry logs reflecting real-world benign activity interleaved with controlled, multi-stage attack scenarios. The foundational goal is to provide empirical substrate for testing detection, forensics, and provenance methods via sequence-based learning and analysis (Riddle et al., 2023).

The second thread, as formalized in "Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI," recontextualizes the Atlas as a rooted hierarchical taxonomy $T = (V, E)$ of single-turn LLM adversarial prompt inputs. Here, the taxonomy operates as both a mapping of adversarial input “styles” and as a guideline for red- and blue-teaming priorities and evaluation workflows (Rawat et al., 2024).

2. ATLAS/ATLASv2: Sequence-Based Attack Investigation Dataset

ATLASv2 advances empirical attack investigation by integrating higher-fidelity background activity and expanded auditing within a controlled lab setting. The pivotal dataset consists of ten “attack engagements” (scenarios s1–s4, m1–m6) executed on up to two Windows 7 hosts (h1, h2). Attacks leverage exploitable CVEs ranging from Adobe Flash (CVE-2015-5122, CVE-2015-3105, CVE-2015-5119) to Microsoft Word (CVE-2017-11882, CVE-2017-0199, CVE-2018-8174).

Each engagement is instrumented with five telemetry streams:

1. Windows ETW Security Auditing
2. Firefox application logs
3. DNS packet captures (Wireshark)
4. Microsoft Sysmon events
5. VMware Carbon Black Cloud sensor data

Total log volume reaches 154 GB with ~8 M Security Auditing events, ~6 M Sysmon events, ~3 M Carbon Black events, ~500k DNS queries, and ~200k Firefox lines. Benign background is generated through “actual user activity,” i.e., two researchers actively use the workstations for standard tasks over four days, with attacks layered during ongoing benign use on day five, producing context-rich, high-fidelity interleaving.

A minimal event schema is enforced for sequence-based learning:

$$\text{Event} = (\texttt{timestamp}, \texttt{host}, \texttt{source}, \texttt{event\_id}, \texttt{process\_name}, \texttt{process\_id}, \texttt{parent\_process\_id}, \texttt{user\_name}, \texttt{session\_id}, \texttt{object\_name}, \texttt{object\_type}, \texttt{additional\_data})$$

Events are mapped to $\mathbf{x}_t \in \{0,1\}^d$ via one-hot or learned embedding; time windows of length $k$ define event sequences $S_t$, which are labeled $y_t$ (attack/benign) for downstream models.

Sequence models such as 1D-CNNs and LSTMs achieve strong baseline metrics (Precision ≈ 0.88–0.90, Recall ≈ 0.84–0.85, F1 ≈ 0.86–0.87), substantiating ATLAS/ATLASv2 as a benchmark for both detection research and forensic investigation workflows (Riddle et al., 2023).

3. Attack Atlas Taxonomy for GenAI Red/Blue Teaming

Within the generative AI domain, the Attack Atlas provides a hierarchical taxonomy of single-turn prompt-based attacks, mapping the design space adversaries inhabit when attempting to subvert LLM safety mechanisms or induce impermissible responses (Rawat et al., 2024). The taxonomy is structured as a rooted tree where nodes represent attack “styles” or “categories.” The top level consists of five pillars:

Table: Five Pillars of Attack Atlas (LLM Domain)

Pillar	Signature Techniques (Abbreviated)	Example Dataset(s)
Direct Instructions	Overt requests for forbidden content; prompt bypass requests	aart, attaq
Encoded Interactions	Obfuscation, payload splitting, output encoding, stylistic evasion	GCG, WordGame, kang2024
Social Hacking	Role-play, social-engineering, persuasion, transcript embedding	jailbreak_prompts, sap
Context Overload	N-shot priming, repeated tokens, irrelevant context flood	Bhatt2024_CyberSecEval
Specialized Tokens	Search/gradient-found token strings triggering unsafe completions	Zou2023_Universal

Each pillar subdivides into attack subtypes (e.g., payload-splitting, role-playing, n-shot priming). Empirical prompts are grouped into these classes based on behavior and semantic features, as validated by intra-group and inter-group embedding similarity analyses.

4. Methodologies for Attack Generation, Detection, and Evaluation

Attack Atlas structures support systematic, reproducible red-teaming as follows:

1. Scoping: Define impermissible behaviors in application context and prioritize Atlas categories based on severity and likelihood.
2. Attack Generation: Populate each pillar with examples drawn from public datasets (e.g., aart for direct instructions) and diversify using automation tools (GCG, TAP, AutoDAN, PAIR, PyRIT). At least N (e.g., 50–200) domain-relevant prompts per category are recommended.
3. Evaluation: Overlay success metrics such as Attack Success Rate (ASR), True Positive Rate (TPr), and False Positive Rate (FPr) on each Atlas leaf. Models evaluated include BERT classifiers, SmoothLLM, and others, with results reported at per-dataset granularity (e.g., BERT TPr: 0.96 on aart, 0.74 on "do not answer"; FPr: 0.01 on alpaca, 0.29 on xstest).
4. Guardrail/Pipeline Design: Multi-stage filtering is advocated—input-stage encoder-only guardrails, semantic classifiers, and fallback LLM refusal models are combined to minimize cost and overblocking.
5. Iteration: Continuous refinement and expansion of attack corpora, with scriptable pipelines for ongoing monitoring and defense evaluation.

A notable conclusion is the lack of intrinsic scoring or probability models within the Atlas; they are externalized into the applied threat model and metric overlays (Rawat et al., 2024).

5. Core Applications and Use Cases

• Forensic Analysis (ATLASv2): Enables replay and analysis of multi-stream system logs for kill-chain reconstruction and provenance.
• Sequence-Based Detection (ATLASv2): Models trained on sequence windows ($S_t$) predict attack/benign state; empirical baselines established.
• Red-Teaming GenAI Systems (Attack Atlas): Structures attack corpora, prioritizes adversarial research and defense, quantifies system robustness per Atlas leaf/branch.
• Blue-Teaming and Guardrail Design: Informs detector and guardrail construction, partitioned by Atlas taxonomy, to reduce overblocking and improve recall on live systems.

6. Challenges, Limitations, and Evolving Threats

Significant challenges span both research threads:

• Red-Teaming Pitfalls: Discrepancies exist between academic attack metrics (ASR-focus) and the practitioner's need to measure actual harm and likelihood. Success criteria (keyword-matching vs. LLM-as-judge) lack consistency, and diversity of attacks is challenging due to automation limitations (Rawat et al., 2024).
• Blue-Teaming Pitfalls: Guardrails are often one-size-fits-all, leading to degraded benign utility when blocking broad Atlas categories (scope drift). The evolving adversarial landscape—emergence of token-smuggling and word-puzzle attacks—necessitates continuous update of detection methods and taxonomies.
• Benchmark Shortcomings: Existing leaderboards omit large segments of the Atlas; high false-positive rates appear on out-of-distribution benign inputs.

This suggests that the Atlas taxonomy and datasets require persistent curation, ongoing adversarial research, and adaptation to emergent attack archetypes.

7. Impact and Outlook

The ATLAS/ATLASv2 and Attack Atlas frameworks represent core empirical and methodological infrastructure for both conventional attack investigation and GenAI adversarial security. They enable standardized evaluation, comprehensive attack coverage, and iterative improvement of both sequence-based and prompt-based detection systems. A plausible implication is that expanding the granularity and automation of Atlas taxonomies will be crucial as new attack modalities and defensive paradigms emerge, solidifying Atlas-driven approaches as central to adversarial research in both classical and modern AI-infused environments (Riddle et al., 2023, Rawat et al., 2024).