Event Proposal Network (EPN)

• Event Proposal Networks (EPNs) are neural modules that generate precise event intervals by predicting temporal and spatial boundaries in detection frameworks.
• They employ architectures such as per-class and single GRUs with specialized loss functions (IoU, focal loss) to enable end-to-end, boundary-aware inference.
• Empirical studies show that EPNs significantly boost detection accuracy and computational efficiency compared to traditional post-processing methods.

An Event Proposal Network (EPN) refers to neural or algorithmic modules designed to predict temporal or spatial regions plausibly containing events of interest, typically as a key component in detection frameworks. EPNs have been independently proposed for both temporal event boundary prediction—particularly in sound event detection (SED)—and for spatial object proposal generation, leveraging event cameras in vision pipelines. The use of “event proposal” terminology unifies threads of research addressing the efficient and precise localization of candidate events around which further inference or refinement can be performed.

1. Integration of Event Proposal Networks in Sound Event Detection

In SED, EPNs are integrated at the core of temporal event modeling pipelines, providing fine-grained, boundary-aware proposals for candidate event intervals. The methodology, as introduced by Schmid, Fouhey, and Kim (Schmid et al., 7 Jan 2026), operates jointly with a Recurrent Event Detection (RED) layer within a hierarchical system:

• The initial acoustic model produces frame-wise logits for each class and time, encoding "event start" and "event end" probabilities.
• The RED layer processes these logits to predict three per-frame, per-class outputs: $\hat p^{\mathrm{pres}}_{c,t}$ (event-presence), $\hat p^{\mathrm{on}}_{c,t}$ (onset), and $\hat p^{\mathrm{off}}_{c,t}$ (offset).
• The EPN consumes these three probability streams, predicting two non-negative durations, $\hat d^{\mathrm{on}}_{c,t}$ and $\hat d^{\mathrm{off}}_{c,t}$, per class and frame. Each pair defines a temporal proposal interval $$\hat r_{c,t} = [t - \hat d^{\mathrm{on}}_{c,t},\, t + \hat d^{\mathrm{off}}_{c,t}]$$, signifying a hypothesized event centered at time $t$.

This explicit modeling of boundary proposals decouples event localization from raw frame-level presence prediction, replacing heuristic smoothing and median filtering with an end-to-end learnable interval proposal mechanism.

2. EPN Architectural Design and Variants

Two principal neural architectures for EPNs in SED are advanced:

• Per-class GRUs: Each class is assigned a two-layer bidirectional GRU, ingesting $[3 \times T]$ input (presence, onset, offset) and emitting $[T \times 2]$ (on/off durations), with a Softplus activation ensuring strictly positive outputs. Parameterization scales with class number (e.g., $\sim$260K parameters for a 10-class set).
• Single GRU: All classes share a single two-layer bidirectional GRU (input $[3|C| \times T]$), producing $[T \times |C| \times 2]$ outputs, particularly for large-scale scenarios (e.g., 4.1M parameters for 447 classes).

Such designs enable EPN modules to learn complex temporal structures in event boundaries, with bidirectionality supporting capture of contextual dependencies.

3. Loss Functions and Optimization Strategies

EPNs employ a boundary-aware optimization objective tailored to interval-level localization:

• Interval IoU Loss: For frames with ground-truth event presence $(y^{\mathrm{pres}}_{c,t} = 1)$, the ground-truth interval $$r_{c,t} = [t - d^{\mathrm{on}}_{c,t},\, t + d^{\mathrm{off}}_{c,t}]$$ is constructed. The loss

$$\mathcal{L}^{\mathrm{IoU}} = \frac{1}{\sum_{c,t} y^{\mathrm{pres}}_{c,t}} \sum_{c=1}^C \sum_{t=1}^T y^{\mathrm{pres}}_{c,t} \frac{1 - \mathrm{IoU}(r_{c,t}, \hat r_{c,t})}{d^{\mathrm{on}}_{c,t} + d^{\mathrm{off}}_{c,t}}$$

penalizes discrepancies between proposal and ground-truth intervals, weighted to equalize contribution across events.

• Focal Losses and Binary Cross-Entropy: Complementary losses include frame-wise binary cross-entropy on event presence predictions and focal losses for onset/offset detection, emphasizing minority/uncertain cases.
• Total Loss: The combined loss is

$$\mathcal{L}^{\mathrm{total}} = \mathcal{L}^{\mathrm{pres}} + \lambda_{\mathrm{ool}} (\mathcal{L}^{\mathrm{on}} + \mathcal{L}^{\mathrm{off}}) + \lambda_{\mathrm{iou}} \mathcal{L}^{\mathrm{IoU}}$$

where hyperparameters $\lambda_{\mathrm{ool}} = 100$, $\lambda_{\mathrm{iou}} \in \{0.5, 1, 2, 4\}$ weight the contributions.

These objectives, coupled with data augmentations such as Freq-MixStyle, filter augmentation, and frequency warping for transformer models, are optimized by AdamW with a cosine learning-rate schedule including 1,000-step warmup periods.

4. Boundary-Aware Inference and Event Selection Procedure

Inference with EPNs departs fundamentally from post-hoc smoothing:

• For each class, mean presence $$\bar p^{\mathrm{pres}}_c = \frac{1}{T} \sum_t \hat p^{\mathrm{pres}}_{c,t}$$ is computed to select the top $m$ classes of interest.
• Within each class, proposals $\hat r_{c,t}$ are sorted by $\hat p^{\mathrm{pres}}_{c,t}$; for each frame, top proposals are iteratively selected, their intervals constructed using the predicted durations and the mean presence $\hat \sigma$ over each interval.
• Overlapping proposals are suppressed by removal. The result is a set of up to $k$ non-overlapping, high-scoring event intervals per class, with $k=15$ and $m=|C|$ used by default.

This boundary-aware selection obviates traditional hand-tuned post-processing, instead directly yielding event intervals with associated confidence scores.

5. Empirical Results and Ablation Insights

Quantitative evaluation demonstrates that EPN-driven approaches provide significant advances in temporal event localization:

Model / Back-end	MF	SEBB	HSM3	Ours (EPN)
CRNN	36.9/30.4	41.1/32.3	39.8/33.2	48.0/40.6
MN-GRU	41.4/33.9	45.7/37.2	45.1/38.8	49.5/42.5
BEATs	48.4/40.2	52.8/44.0	52.5/44.5	55.2/46.7
ATST-F	48.2/39.9	51.9/42.4	52.3/44.9	56.6/48.9

Table: PSDS1/P200 ms-F1 metrics on AS-Strong-10; "Ours" refers to RED+OOL+EPN pipeline.

Ablation studies attribute improvements to (a) focal loss on onsets/offsets and (b) the inclusion of EPN with boundary-aware inference, driving superior results over traditional median filtering and state-of-the-art post-processing such as SEBB and HSM3. On the AudioSet-Strong-Full benchmark, EPN-based models yield a new state-of-the-art PSDS1 of 49.6 with BEATs and 47.7 with ATST-F, exceeding competing ensemble distillation pipelines. The configurable architecture allows the EPN approach to match high-capacity models even with lightweight CRNNs, emphasizing the efficacy of learned interval proposals.

6. Event Proposal Networks with Event Cameras for Region Proposals

In parallel, EPNs have been proposed to address spatial region proposal generation in computer vision by leveraging streams from event cameras (Awasthi et al., 2023). Here, the standard region proposal network (RPN) stage of two-stage detectors (such as Mask R-CNN) is supplanted by a low-latency, unsupervised event-based clustering module:

• Event cameras output asynchronous tuples $e_i = (x_i, y_i, t_i, p_i)$, marking pixel-level brightness changes.
• Accumulated events within a short time window are rasterized into a 2D pseudo-image, denoised via morphological erosion, and clustered (DBSCAN).
• Bounding boxes are drawn tightly around each cluster, yielding $K \leq 10$ proposals reflecting the number of moving objects rather than a dense, anchor-based grid.
• Proposals are fed to the original ROI head for classification and refinement.

This configuration reduces the typical $1,000$ RPN proposals per image to $K\approx \#$moving-objects per frame, offering substantial computational savings.

7. Quantitative Evaluation, Limitations, and Research Directions in Event-Based EPNs

Empirical comparisons on several RGB+event video sequences show that using event-camera–driven EPN modules achieves a mean average precision (mAP) at $\mathrm{IoU} \geq 0.75$ within $\sim$5.4% of Detectron2’s baseline (92.78 vs. 87.39), while drastically reducing downstream computational burden. DBSCAN was chosen for clustering based on superior efficiency over graph-spectral alternatives, with morphological erosion (kernel size $3 \times 3$) suppressing noise.

Limitations include the need for stationary-camera setups (as ego-motion introduces background events), reliance on classical (non-learned) clustering over event streams, approximate cross-modal calibration, and the absence of end-to-end joint training with ROI heads. Future work intends to explore lightweight event-CNNs for proposal generation and expanded evaluation on large-scale benchmarks and dynamic scenes, suggesting that direct event-driven region proposal may serve as a foundation for new hybrid detection pipelines in both vision and temporal sequence domains.

---

References:

• (Schmid et al., 7 Jan 2026) Sound Event Detection with Boundary-Aware Optimization and Inference
• (Awasthi et al., 2023) Event Camera as Region Proposal Network