Phase Aware Attention Module

• Phase aware attention is a neural mechanism that integrates phase signals (Fourier, temporal, or learned) to guide attention weights.
• The module employs distinct pathways for phase and magnitude processing, enabling precise extraction of structural and discontinuous features.
• Applications in text comprehension, deepfake detection, and multi-phase CT imaging consistently show improved accuracy and robustness.

A phase aware attention module is a class of neural attention mechanisms that exploits phase or phase-like information (from signal, feature, or sequence perspectives) within the attention computation to better capture relationships in structured, multi-phase, or frequency-rich data. Such modules have emerged in text machine comprehension, vision, frequency-domain forensics, multi-modal medical imaging, and positional encoding for transformers, with diverse realizations adapted to their target domains.

1. Principle and Definitions

Phase aware attention refers to attention mechanisms that explicitly utilize phase information—whether physical (Fourier phase), sequence-based (modality/phase indices), or learned phase functions—to guide the allocation of attention weights between elements of the input. In practice, "phase" may denote:

• True spectral phase, extracted from a Fourier transform for frequency-domain data.
• Temporal, modality, or scan phase, as in multi-phase medical imaging.
• Token-specific learnable phase, as in sequence models.

The core design contrasts with conventional attention (which typically operates on amplitudes, magnitudes, or feature activations) by weighting dependencies in a manner sensitive to phase structure, discontinuities, or cross-phase relations.

2. Architectures and Operational Modalities

2.1 Multi-Phase Attention in Sequential Architectures

PhaseCond (Liu et al., 2017) decomposes multi-layer attention for machine comprehension into two phases: (i) question–passage cross-attention and (ii) self-attention (evidence propagation) on the passage. Each phase consists of attention layers interleaved with fusion/gating sublayers. Cross-phase dependency is interpreted structurally—each "phase" aligns with a computational or semantic processing phase, rather than a signal-theoretic phase.

2.2 Frequency-Domain Phase Attention

Phase4DFD (Lin et al., 9 Jan 2026) introduces a Phase-Aware Attention Module (PAAM) at the input level for deepfake detection. After augmenting RGB data with FFT magnitude and local binary pattern (LBP) channels, PAAM computes an attention map from both the normalized Fourier phase and magnitude. The attention mechanism operates by extracting phase discontinuities from the FFT phase spectrum and integrating these as features alongside magnitude, passing them through parallel convolutional branches before fusing via a sigmoid attention map.

2.3 Inter-Phase Attention in Medical Imaging

LACPANet (Uhm et al., 2024) computes attention across scan phases (e.g., non-contrast, arterial, portal, delayed) in multi-phase CT. The inter-phase attention module pools lesion-centric feature vectors per phase, enriches them with learnable phase (i.e., scan-phase) embeddings, and then computes an $N \times N$ attention matrix where each entry quantifies how features from phase $j$ should modulate those in phase $i$ through scaled dot-product attention and residual cross-phase mixing.

2.4 Phase-Based Self-Attention in Transformers

Phaseformer (Khan et al., 2024) and Token-Aware Phase Attention (TAPA) (Yu et al., 16 Sep 2025) generalize phase aware attention to transformer-style self-attention mechanisms. In Phaseformer, the attention head projects the input feature maps into the frequency domain, discards amplitude (retaining only the complex phase), and then operates phase-only attention. TAPA replaces Rotary Positional Embedding (RoPE)'s static token-agnostic phase with a learnable, token-aware phase function, producing a content-sensitive, distance-dependent cosine phase modulation inside the attention score.

3. Mathematical Formulations

Phase aware attention modules instantiate diverse mathematical forms depending on the context:

Frequency Domain Phase Attention

Given FFT phase $\phi$ and magnitude $M$ extracted from an image, PAAM (Lin et al., 9 Jan 2026) processes each via separate convolutional pathways, concatenates activations, and projects to an attention map $A_p \in \mathbb{R}^{5 \times H \times W}$, applied to the augmented input via elementwise multiplication before downstream processing:

$$A_{p} = \sigma\left(\text{Conv}_{1 \times 1}(\text{ReLU}(\text{BN}([\text{Conv}_{3\times 3}(M), \text{Conv}_{3\times 3}(P)])))\right)$$

$X^\prime = X^0 \odot A_{p}$

Inter-Phase Attention (Medical Imaging)

With per-phase, lesion-aware query, key, value vectors $Q_i, K_i, V_i \in \mathbb{R}^C$ and learnable phase embeddings $P_i \in \mathbb{R}^C$:

$$\tilde{Q}_i = Q_i + P_i\,, \quad \tilde{K}_i = K_i + P_i\,, \quad \tilde{V}_i = V_i + P_i$$

$$A = \operatorname{softmax}\left(\frac{QK^T}{\sqrt{C}}\right) \in \mathbb{R}^{N \times N}$$

$F_{\mathrm{out}} = V + \lambda\,A\,V$

Phase-Based Self-Attention (Transformers)

For input $Y \in \mathbb{R}^{H' \times W' \times C'}$, after phase extraction:

$P = \mathrm{IFFT}(e^{j\phi_f})$

With pointwise and depthwise convolutions, the attention weights are computed from phase-only projections. The attention map is:

$$A = \operatorname{Softmax}\left(\frac{1}{\alpha}\hat{\Phi}_k \hat{\Phi}_q\right)$$

Token-Aware Phase Attention

In the transformer setting with $q, k \in \mathbb{R}^D$, learnable function $\varphi$, and sequence indices $(m,n)$:

$$\operatorname{Attn}_{\theta, \alpha}(q, k) = \frac{q_A^\top k_A}{\sqrt{\theta D}} \cdot \cos\left(2 \pi |m - n|^\alpha \frac{q_P^\top k_P}{\sqrt{(1-\theta)D}}\right)$$

(Yu et al., 16 Sep 2025)

4. Key Implementation Strategies and Mechanisms

Gating and Fusion

Stacked attention layers are interleaved with fusion modules (outer fusion for concatenation across attention layers, inner fusion as a GRU-like gate in recurrent self-attention), as in PhaseCond (Liu et al., 2017). In frequency or feature domains, attention outputs are fused multiplicatively or via channel gating.

Learnable Phase Embeddings

Introducing per-phase (or per-modality) learnable embeddings helps preserve explicit phase identity in cross-phase attention, improving performance in tasks with otherwise ambiguous phase semantics (Uhm et al., 2024).

Phase-Only or Phase-Magnitude Processing

Selective use of phase or phase-magnitude combinations (e.g., discarding amplitude as in Phaseformer (Khan et al., 2024), or combining both in PAAM (Lin et al., 9 Jan 2026)) allows the module to focus on structural features and phase discontinuities known to be diagnostic in specific application domains.

Multi-Scale and Multi-Head Extensions

Spatial and feature-wise multi-scale attention, as well as multi-head structures with per-head phase processing, have been demonstrated to improve model expressivity and robustness, particularly in high-resolution vision or 3D imaging tasks (Uhm et al., 2024, Khan et al., 2024).

5. Empirical Impact Across Domains

Empirical results show phase aware attention mechanisms consistently yield performance gains over both magnitude-only and generic attention baselines in diverse settings:

Paper / Module	Domain	Metric / Dataset	Notable Gains
PhaseCond (Liu et al., 2017)	NLP (QA)	SQuAD EM/F1	+4–5 EM/F1 over BiDAF, +0.23 F1 over multi-layered models
PAAM (Lin et al., 9 Jan 2026)	Deepfake detection	DFFD Accuracy	+0.23pp over RGB, +0.33–0.35pp over FFT/LBP baselines
LACPANet (Uhm et al., 2024)	Medical (CT)	AUC	+0.0407 AUC (attention), +0.0215 (multi-scale), +0.0582 total vs. baseline
Phaseformer (Khan et al., 2024)	Image Restoration	PSNR/SSIM/UIQM (UIEB/UFO-120)	+1.74 PSNR, +0.066 SSIM, +0.725 UIQM over best prior SOTA
TAPA (Yu et al., 16 Sep 2025)	LLM positional encoding	PG19 PPL, 64k context	−9.4% PPL at 32k, stable at 64k where baselines collapse

Phase aware modules consistently outperform standard attention, CBAM, and frequency-domain magnitude-only models in their respective evaluation protocols.

6. Theoretical Properties and Guarantees

Phase aware attention modules introduce distinctive theoretical properties:

• In TAPA (Yu et al., 16 Sep 2025), the token-aware phase function guarantees vanishing mean attention bias with increasing sequence distance, in contrast to token-agnostic phases (RoPE), which accumulate systematic long-distance bias. Crucially, TAPA preserves nondegenerate variance, so long-range information is not lost.
• For multi-phase imaging and deepfake detection, explicit modeling of phase discontinuities and inter-phase dependencies both enables detection of otherwise elusive artifacts and provides a mechanism for cross-domain generalization.

7. Adaptations, Limitations, and Application Guidelines

• Masking or pooling keyed to explicit regions of interest (e.g., lesion masks in CT) ensures that phase aware attention remains localized to semantically relevant regions (Uhm et al., 2024).
• Learnable phase (or modality) embeddings are advantageous in settings with ambiguous or redundant phase information.
• The core design is modular and lightweight, permitting integration with CNNs, transformers, or hybrid models with minimal FLOP/parameter overhead, as evidenced by the modest computational increases in PAAM and LACPANet.
• Multi-scale or multi-head variants are most beneficial when phase effects are heterogeneous across resolution or modality.

Phase aware attention modules thus summarize a broad family of mechanisms that instantiate phase-sensitive dependencies for improved modeling in structured, sequence, or frequency-based data across NLP, vision, and medical analysis domains (Liu et al., 2017, Lin et al., 9 Jan 2026, Uhm et al., 2024, Khan et al., 2024, Yu et al., 16 Sep 2025).