Intrinsic Risk Sensing in LLM Agents

• Intrinsic Risk Sensing is an internal mechanism that monitors agent-generated queries, plans, actions, and observations to trigger stage-specific security routines.
• It employs a two-tier screening approach, using fast cosine-similarity matching for most cases and deep LLM analysis for ambiguous threats.
• IRS integration in LLM agents reduces attack success and false positive rates while minimizing operational overhead compared to always-on security pipelines.

Intrinsic Risk Sensing (IRS) is an agent-internal vigilance mechanism that enables event-driven, stage-specific security checks in LLM agents. IRS continuously monitors critical agent lifecycle artifacts—user queries, plans, actions, and observations—and selectively triggers hierarchical defense routines only when a credible threat is internally recognized. Unlike mandatory, externally imposed security pipelines, IRS operationalizes risk perception as an endogenous, interrupt-driven cognitive function, balancing precision, latency, and coverage with minimal operational overhead (Yu et al., 5 Feb 2026).

1. Formal Definition and Operational Distinction

IRS in Spider-Sense is defined as a built-in, selective vigilance process that evaluates agent-generated artifacts at four security-critical stages: query, plan, action, and observation. At each step $t$ and stage $k$, the agent computes the conditional probability

$P(\varphi_t^{(k)}=1 \mid h_{t-1},\,p_t^{(k)},\,I),$

where $h_{t-1}$ is the interaction history, $p_t^{(k)}$ the artifact, and $I$ the high-level system instruction. A security check is triggered only if

$$\varphi_t^{(k)} = \begin{cases} 1, & \text{if } P(\varphi_t^{(k)} = 1 \mid h_{t-1}, p_t^{(k)}, I) > \sigma^{(k)} \ 0, & \text{otherwise} \end{cases}$$

with threshold $\sigma^{(k)}$ controlling sensitivity at each stage. Operationally, the agent emits a stage-specific template (e.g., <|verify_user_intent|> ... </|verify_user_intent|>) precisely at the point of perceived risk, thereby internalizing the defense trigger. In contrast, mandatory decoupled defenses invoke an external verifier at every fixed stage, disregarding actual context and yielding increased latency and false positives.

2. Event-Driven Architecture and Latent Vigilance

IRS adopts an event-driven, interrupt-responsive architecture. The agent proceeds as follows:

1. Reason and generate stage-$k$ artifact $p_t^{(k)}$.
2. Internally compute if $\varphi_t^{(k)} = 1$ via the calibrated probability.
3. If $\varphi_t^{(k)}=0$, the workflow continues unimpeded.
4. If $\varphi_t^{(k)}=1$, the artifact is wrapped in the corresponding template and passed to hierarchical defense.

This latent vigilance ensures that the agent is perpetually ready to intercept threats but initiates costly screening only under concrete risk. The result is the minimization of unnecessary interruptions and resource expenditure for benign interactions.

3. Hierarchical Adaptive Screening (HAC)

Upon risk detection, the IRS mechanism dispatches the artifact to a Hierarchical Adaptive Screening inspector tailored to each stage. HAC operates in two phases:

• Coarse-Grained Detection (Similarity Matching):
  • Each stage $k$ maintains a vector-indexed case bank

  $$\mathcal{D}^{(k)} = \{ (\mathbf{v}_i^{(k)}, z_i^{(k)}, d_i^{(k)}) \}_{i=1}^{N_k}$$

  where $\mathbf{v}_i^{(k)}$ are attack embeddings, $z_i^{(k)}$ metadata, $d_i^{(k)}$ verdicts. - Compute cosine similarity between the input embedding $\mathbf{v}_t^{(k)}$ and stored cases.

  $$s_t^{(k)} = \max_i \cos(\mathbf{v}_t^{(k)}, \mathbf{v}_i^{(k)})$$ - If $s_t^{(k)} \ge \tau^{(k)}$, the matched verdict is returned at minimal cost ($T_\mathrm{coarse}$, order tens of ms).

• Fine-Grained Analysis (Deep Reasoning):
  • For $s_t^{(k)} < \tau^{(k)}$, retrieve Top-$K$ nearest patterns and invoke an LLM-based reasoner:

  $$\mathbf{r}_t^{(k)} = \mathcal{R}_\mathrm{LLM}(p_t^{(k)}, \mathcal{N}_t^{(k)})$$

  producing Accept/Reject/Sanitize decisions with rationale at cost $T_\mathrm{fine}$ (typically 0.5–1 s). - The design allows for the automated balancing of efficiency (low $\delta$, the fraction escalating to fine analysis) and threat coverage by tuning $\tau^{(k)}$.

A table summarizes HAC stages and processing flow:

Stage	Operation	Efficiency/Precision Trade-off
Similarity Matching	Cosine search in $\mathcal{D}^{(k)}$	$T_\mathrm{coarse}$, high efficiency
Deep Reasoning (LLM)	Verdict on ambiguous cases	$T_\mathrm{fine}$, high precision

A plausible implication is that the vast majority of incidents are resolved by the fast path, with only rare or novel threats requiring expensive LLM reasoning.

4. Benchmarking and Performance Metrics (S²Bench)

Evaluation employs the S²Bench benchmark, encompassing all four agent stages, with 79 sub-scenarios over eight mission domains (IT, finance, healthcare, etc.), featuring:

• Attacks per stage: Query (76), Plan (123), Action (134), Observation (104)
• 153 hard benign prompts for rigorous FPR estimation
• Active Attack Simulation Injector for lifecycle-accurate testing (300+ tool functions, 100+ return types)

Key metrics:

• Attack Success Rate (ASR): $$\frac{\#\,\mathrm{successful\,attacks}}{\#\,\mathrm{malicious\,attempts}}$$
• False Positive Rate (FPR): $$\frac{\#\,\mathrm{benign\ cases\ incorrectly\ blocked}}{\#\,\mathrm{benign\ cases}}$$
• Attack Mitigation: $1 - \mathrm{ASR}$
• Label Prediction: Accuracy, Precision, Recall, F1

Experimental highlights with Qwen-Max as the base model:

• Stage-wise ASR: (11.9%, 20.0%, 10.8%, 11.0%), total ASR: 13.6%
• FPR: 10.4% (lowest among baselines)
• End-to-end runtime: 23.4 s vs. 21.6 s baseline ($+8.3\%$ overhead)

Interpretation: IRS triggers HAC on approximately 24% of steps, with the majority handled by similarity matching. Most steps incur no screening cost, and deep analysis is reserved for ambiguous inputs. Always-on pipelines, by comparison, induce greater than 200% overhead, underscoring IRS’s efficiency.

5. Implementation and Integration Guidelines

Adopting IRS in other LLM agents involves:

1. Instruction-Level Conditioning: Extend system prompts to enable emission of risk tags (e.g., <|audit_action_parameters|>) when suspicious patterns arise.
2. Stage-wise Interfaces: Clearly differentiate four artifact entry points—query, plan, action, observation—with dedicated triggering templates.
3. Threshold-Tuned Vector Stores: Build high-quality, stage-specific attack pattern banks; use universal encoders (e.g., BGE-M3); calibrate $\tau^{(k)}$ to maintain fine-analysis fraction $\delta<0.3$.
4. Two-Tier Screening: Apply cosine similarity checks by default, escalating only low-confidence matches to LLM-based reasoning.
5. Lifecycle-Aware Testing: Use comprehensive, multi-stage benchmarks (e.g., S²Bench) to identify gaps and test for compositional vulnerabilities.
6. Continual Refinement: Update vector banks via Refine-and-Filter methodology to adapt to evolving attacks without retraining the base agent.

IRS effectively recasts security from a peripheral, high-cost function into an efficient, cognitively integral property of LLM agents, yielding robust threat mitigation with practical end-to-end performance (Yu et al., 5 Feb 2026).

6. Context, Significance, and Future Directions

IRS marks a paradigm shift from static, externally policed verification toward internalized, selective defense in modern agent systems. By coupling event-driven vigilance with hierarchical, adaptive screening, IRS achieves both substantial reductions in attack success and false positive rates and markedly lower runtime impact relative to traditional always-on defenses.

A plausible implication is that IRS’s architectural principles generalize across agent types and domains, provided robust artifact typing and pattern bank management. Lifecycle-aware evaluation, as exemplified by S²Bench, is essential for fully characterizing IRS performance, particularly under complex, multi-phase attack vectors.

Ongoing research directions include dynamic adaptation of sensitivity thresholds ($\sigma^{(k)}$, $\tau^{(k)}$), automated pattern bank updating, and further reduction of fine-screening latency. As agents grow in autonomy and operational scope, IRS embodies a fundamental approach for scalable, context-sensitive security.