Transformer-Based Ear Recognition Systems

• Transformer-based ear recognition systems are advanced biometric frameworks that leverage self-attention and innovative tokenization strategies for precise identification.
• Incorporating low-rank linear layers reduces parameters by up to 29% and enables real-time, edge-device deployment without sacrificing performance.
• Diffusion-based inpainting effectively handles occlusions, significantly improving AUC metrics in unconstrained and challenging biometric scenarios.

Transformer-based ear recognition systems are advanced computer vision frameworks that leverage self-attention architectures—most notably Vision Transformers (ViTs) and their hybrids—for unconstrained ear biometric identification and verification. These models have demonstrated substantial gains over traditional CNN-based systems by exploiting global context modeling, overlapping/pseudo-anatomic patch strategies, lightweight low-rank adaptations, and pre-processing with generative inpainting to increase accuracy, generalizability, and efficiency in challenging biometric scenarios (Lendering et al., 11 Feb 2025, Arun et al., 30 Mar 2025, Arun et al., 27 Jan 2026, Arun et al., 27 Jan 2026).

1. Architectural Foundations and Evolution

Early transformer-based ear recognition approaches primarily adapted canonical ViTs to domain-specific constraints. A typical pipeline involves: (i) side-classification (to distinguish left/right ears), (ii) fixed-size cropping and normalization, (iii) patch-based tokenization, (iv) Transformer encoder processing, and (v) metric-space embedding for matching.

Notable evolutionary trends include:

• Hybrid CNN–Transformer backbones: EdgeEar deployed a shallow CNN "stem" followed by a few Split Depth-wise Transpose Attention (SDTA) transformer blocks. This efficiently balances localized detail extraction with global context modeling, reducing critical overparameterization typical in vanilla ViTs (Lendering et al., 11 Feb 2025).
• Lightweight architectures: Parameter-efficient low-rank factorization (see Section 2) and sub-2M parameter footprint models make deployment on edge devices tractable.
• Adaptation to biometric domain: Both completely raw and anatomy-aware tokenizations have been explored to accommodate the complex geometry and variable occlusion rates characteristic of ear images (Arun et al., 27 Jan 2026).

2. Parameter Efficiency: Low-Rank Linear Layers

Resource constraints—particularly on embedded and edge devices—require substantial reductions in parameter count and computation. Low-rank approximation of transformer linear layers (the "LoRaLin" technique) decomposes each weight matrix $W\in\mathbb{R}^{M\times N}$ into two slim factors $A\in\mathbb{R}^{M\times r}$, $B\in\mathbb{R}^{r\times N}$, with $r \ll \min(M, N)$ and $r = \max(2, \gamma \cdot \min(M, N))$ where $\gamma \in (0, 1]$ is a hyperparameter. Applied selectively—predominantly to Q, K, V projections in attention blocks—this yields up to 29% parameter reduction and 24% FLOP savings with negligible or even improved EER on even unconstrained datasets (Lendering et al., 11 Feb 2025). Overaggressive reduction ($\gamma < 0.4$) degrades EER but moderate settings maintain representation power, enabling real-time inference (4–11 ms/image) on ARM CPUs and embedded GPUs.

Model Component	Full-rank Params	LoRaLin Params	FLOPs Reduction
EdgeEar Backbone	2.80M	1.98M	24%

3. Patch Tokenization Strategies: Overlapping and Anatomy-Aware

Overlapping Patches

Most transformer architectures use a non-overlapping, grid-based tokenization. However, Arun et al. established that overlapping patch extraction (stride $S < P$, typically $S = P/2$ for 50% coverage) is critical for capturing the contiguous curves and fine-grained morphological cues of the ear—particularly helix, antihelix, and concha structures (Arun et al., 30 Mar 2025). Their comprehensive benchmark found that overlap improved AUC in 44/48 cases, yielding up to a 10% relative gain (EarVN1.0, $P=56$, $S=28$).

Patch Size (P), Stride (S)	Overlap	EarVN1.0 AUC (ViT-T)
16, 16	0%	0.7726
28, 14	50%	0.7838
56, 28	50%	0.6966 (+10.1%)

Optimal balance is achieved with $P=28$, $S=14$ for both efficiency and discriminatory power.

Anatomy-Guided Warping

PaW-ViT introduced a preprocessing method that leverages segmentation maps or keypoint landmarks to warp and partition the ear region into radial sectors matching anatomically meaningful boundaries. Each warped quadrilateral is affinely mapped to a square patch (typically $16 \times 28 \times 28$ grid), maintaining continuity of features across tokens and suppressing background distractions (Arun et al., 27 Jan 2026). This approach, combined with off-the-shelf ViTs, provided robust performance gains (+4.6% AUC on EarVN1.0) and enhanced generalization, especially under large inter-subject shape or pose variation.

4. Occlusion Handling via Diffusion-Based Ear Inpainting

Occlusions from earrings, headphones, or hair present a significant challenge. "Diffusion for De-Occlusion" introduced an accessory-aware diffusion inpainting module, initiated by an automated YOLOv10 + Grounding DINO + SAM2-based mask generation pipeline (Arun et al., 27 Jan 2026). The U-Net-based DDPM fills occluded regions before recognition. Baseline and inpainted pipelines—trained and evaluated separately—demonstrate that inpainting yields substantial improvements on occlusion-heavy datasets (e.g., EarVN1.0: baseline 0.7086 $\rightarrow$ inpainted 0.7660 AUC for ViT-B, $p=16$), with the most pronounced benefits observed at coarse patch sizes and in highly unconstrained conditions.

Deployment consideration: inpainting introduces $\sim$1s latency per image, motivating selective invocation only on occlusion-flagged samples.

5. Benchmarks, Experimental Protocols, and Performance Comparison

Systems are evaluated using large, diverse benchmarks comprising UERC2023 (training), OPIB, AWE, WPUT, and EarVN1.0 (testing), capturing a spectrum of ethnicity, pose, occlusion, and capture conditions (Lendering et al., 11 Feb 2025, Arun et al., 30 Mar 2025, Arun et al., 27 Jan 2026, Arun et al., 27 Jan 2026). Metrics include Equal Error Rate (EER), Area Under the ROC Curve (AUC), and computational cost (parameters, FLOPs, latency).

Key empirical findings:

• EdgeEar: EER $=0.143$ (lowest in class), AUC $=0.904$, 11 ms/image ARM CPU (Lendering et al., 11 Feb 2025).
• ViT-Tiny with overlapping patches: AWE (AUC$=0.9834$), WPUT ($0.9472$), EarVN1.0 ($0.7838$), outperforming larger backbones (Arun et al., 30 Mar 2025).
• PaW-ViT with "Union" warping: ViT-T AUC on EarVN1.0 from 0.7356 (raw) to 0.7820 (warped) (Arun et al., 27 Jan 2026).
• Diffusion inpainting: ViT-B AUC on EarVN1.0 from $0.7086\pm0.0118$ (baseline) to $0.7660\pm0.0040$ (inpainted) (Arun et al., 27 Jan 2026).

6. Trade-Offs, Limitations, and Future Directions

Trade-offs and open challenges include:

• Model complexity vs. accuracy: Low-rank and overlapping/warped tokenization maintain or improve accuracy at drastically reduced parameter budgets, enabling edge deployment.
• Over-approximation risks: Excessive rank reduction in LoRaLin ($\gamma < 0.4$) impairs performance (Lendering et al., 11 Feb 2025).
• Segmentation/inpainting fidelity: Performance is sensitive to the quality of anatomical masks and accessory masks; misfires degrade recognition (Arun et al., 27 Jan 2026, Arun et al., 27 Jan 2026).
• Generalizability: Warping and overlap improve robustness under pose, scale, and occlusion, but do not fully resolve outlier or rare morphology cases.
• Computational bottlenecks: Diffusion methods introduce perceptible latency, but are justified where robust occlusion removal is critical.

Recommendations from the literature point toward:

• Multi-scale/patch-size hybridization.
• End-to-end joint optimization of segmentation, warping, and recognition.
• Identity-preserving regularization in generative pre-processing.
• Broader modality fusion (ear+face/iris) for multi-biometric authentication (Lendering et al., 11 Feb 2025).

7. Practical Deployment and Extensions

Transformer-based ear recognition systems have reached a level of accuracy and computational efficiency compatible with mobile and embedded applications. EdgeEar achieves $\sim$2M parameters and $<$15 ms inference, enabling biometric solutions for low-power devices (Lendering et al., 11 Feb 2025). Quantization and pruning are proposed to further reduce latency (<5 ms) (Lendering et al., 11 Feb 2025).

The field is now exploring:

• Occlusion detection "gating" to minimize unnecessary pre-processing (Arun et al., 27 Jan 2026).
• On-device domain adaptation via fine-tuning of select layers.
• Cross-modality fusion and late-stage ensemble frameworks.
• Transfer of warping and overlapping patch paradigms to related biometric domains (face, iris) (Arun et al., 27 Jan 2026).

Transformer architectures, when paired with domain-specific tokenization and generative enhancement strategies, now represent the vanguard of robust, efficient, and scalable ear biometric recognition.

---

References:

• (Lendering et al., 11 Feb 2025) EdgeEar: Efficient and Accurate Ear Recognition for Edge Devices
• (Arun et al., 30 Mar 2025) Improved Ear Verification with Vision Transformers and Overlapping Patches
• (Arun et al., 27 Jan 2026) PaW-ViT: A Patch-based Warping Vision Transformer for Robust Ear Verification
• (Arun et al., 27 Jan 2026) Diffusion for De-Occlusion: Accessory-Aware Diffusion Inpainting for Robust Ear Biometric Recognition