ISP-Based Flicker Simulation Pipeline

• The ISP-based flicker simulation pipeline is a modular signal processing framework that isolates and simulates non-stationary signal modes using time-frequency representations.
• It employs a non-iterative architecture with STFT, adaptive thresholding, and image segmentation to robustly extract overlapping, high-energy regions despite high noise levels.
• The pipeline supports practical applications such as structural health monitoring by accurately reconstructing modal signals from complex data.

An ISP-based flicker simulation pipeline is a modular signal processing architecture for isolating and analyzing the constituent components—or "modes"—of non-stationary and multicomponent temporal signals, using time-frequency domain representations. In the context of modern computational signal analysis, pipelines of this kind facilitate adaptive, robust, and efficient decomposition of complex input data, with particular focus on separating overlapping or closely-spaced phenomena that are otherwise challenging to extract using traditional iterative methods. A representative implementation can be found in the Time-Frequency Mode Decomposition (TFMD) framework, which leverages the Short-Time Fourier Transform (STFT) and image segmentation techniques for mode extraction (Zhou et al., 16 Jul 2025).

1. Principles of Time-Frequency Decomposition

TFMD operates on the premise that coherent signal modes manifest as contiguous high-energy regions in the time-frequency (TF) domain, a fact that justifies reformulating the decomposition problem as an image segmentation task over a smoothed spectrogram. Unlike parametric or physics-informed approaches that require explicit knowledge of the number, bandwidths, or central frequencies of modes, TFMD adaptively discovers such information from the data. This eliminates the trade-offs between accuracy and computational burden inherent in iterative time-domain approaches such as Empirical Mode Decomposition (EMD), Adaptive Chirp Mode Decomposition (ACMD), and Variational Mode Decomposition (VMD).

Key to this approach is the use of a non-iterative, pipeline-style design where the signal $x[n]$ undergoes the following transformations:

1. STFT: Windowed time segments are transformed into the frequency domain using a centered-frequency mapping to ensure consistent spectral alignment.
2. Spectrogram Smoothing: A local averaging filter (typically $3\times3$ moving-average) is applied to the magnitude spectrogram to enhance blob continuity and suppress isolated spectral noise.
3. Adaptive Thresholding: A data-driven threshold $\tau$ derived from the geometric mean of the global maximum and median is used to binarize the spectrogram.
4. Connected-Component Labeling (CCL): Binary masks are generated for contiguous high-energy regions, with subsequent size-based filtering to discard spurious or noise-induced artifacts.
5. Mode Mask Generation: Each validated region defines a binary mask, which when applied to the original STFT matrix, isolates its spectral contribution.
6. Reconstruction (ISTFT): Inverse STFT and overlap-add reconstruct the time-domain signal for each mode.

2. Core Module Workflow

The TFMD pipeline is realized via the following algorithmic steps:

• STFT Computation:

$$S[k, m] = \sum_{n=-\infty}^{\infty} x[n]\,w[n-(m-1)R]\,e^{-j2\pi\,F(k) n/f_s}$$

with frequency bin mapping $F(k) = (k - k_{\text{DC}})\,f_s/N$ and hop size $R$.

• Spectrogram Smoothing:

$$\hat{S}[k', m] = \frac{1}{|\mathcal{N}_{U\times V}|} \sum_{(u,v)\in\mathcal{N}_{U\times V}(k', m)} |S^{+}[u, v]|$$

• Global Adaptive Threshold:

$$\tau = \sqrt{ \frac{ \max_{k', m} \hat{S}[k', m] \times \text{median}(\hat{S}) }{ C_\text{thresh} } }$$

($C_\text{thresh}=2$ in practice)

• Binary Masking and CCL:

$$B_{\text{ini}}[k', m] = \begin{cases} 1 & \hat{S}[k', m] > \tau\ 0 & \text{otherwise} \end{cases}$$

CCL operates via two-pass union-find to assign labels to connected "blob" regions.

• Size-Based Filtering: Isolates the largest modes based on absolute and relative pixel criteria.
• Mode Mask Extension and ISTFT: Each mask is symmetrically extended for real-valued signals, multiplied with $S[k,m]$, and inverse-transformed with the original synthesis window and hop size.

3. Computational Complexity and Robustness

The pipeline's non-iterative architecture is dominated by FFT-based operations, with total complexity scaling as $O(N_f\,T\log L_w)$ for a signal of length $T$ and $N_f$ recovered modes. Each step—spectrogram smoothing, mask extraction, and CCL—admits linear or quasi-linear cost. By comparison, classical methods typically scale as $O(K \cdot I \cdot T)$ ($K$=number of modes, $I$=iterations per mode) or $O(T^2)$.

TFMD demonstrates robust performance under high noise conditions (input SNR $<$ 25 dB), where binary masking effectively eliminates noise outside dominant time-frequency regions. A persistent reconstruction error floor is observed at very high input SNR, attributed to binary masking and the overlap-add procedure. Critically, TFMD consistently determines the correct number of modes even in highly corrupted data.

4. Application: Structural Health Monitoring

In real-world settings, TFMD extends to modal analysis tasks crucial to structural health monitoring. For instance, in analyzing Dowling Hall footbridge vibration data, TFMD extracted four modes whose peak frequencies precisely matched the physical modal frequencies. The sum of reconstructed modal signals maintained the modal peaks while attenuating broadband noise, directly supporting applications that require denoising and modal feature isolation (Zhou et al., 16 Jul 2025).

5. Comparison with Other Decomposition Strategies

TFMD holds several principal advantages:

Method	Adaptivity	Iterations	Parameter?	Computational Cost	Mode-Distinguishing
TFMD (Zhou et al., 16 Jul 2025)	Yes	None	None	$O(N_f\,T\log L_w)$	Strong
EMD/ACMD	Partial	Yes	# Modes	$O(K\cdot I\cdot T)$	Moderate
VMD	Partial	Yes	# Modes	$O(K\cdot I\cdot T\log T)$	Good
SET/VGNMD	Partial	Yes	# Modes	$O(T^2)$	Strong

TFMD's pipeline obviates the need for prior mode specification, iterative sifting, or global optimization, and its use of adaptive segmentation and thresholding directly responds to the data's time-frequency structure.

6. Integration and Practical Considerations

TFMD can be implemented efficiently in any numerical environment (e.g., Python, MATLAB) as all steps require only core vectorized operations, local smoothing, and feature labeling. Key parameters such as window length, hop size, and smoothing kernel are straightforward to tune based on the input signal's characteristics, but TFMD's performance is not tightly sensitive to their precise values. Mode summation directly reconstructs the signal, and all steps preserve the original signal length and structure.

The pipeline's reliance on image segmentation rather than parameter-fitting or heuristic band isolation is a major point of departure from other frameworks. It facilitates rapid deployment for large-scale or real-time applications where interpretability, speed, and robustness to noise are required (Zhou et al., 16 Jul 2025).

---

References

• "STFT-based Time-Frequency Mode Decomposition: A Fast and Robust Method for Multicomponent Signal Analysis" (Zhou et al., 16 Jul 2025).