Neural Directional Filtering (NDF)

• Neural Directional Filtering (NDF) is a deep neural network-based spatial audio filtering method that infers time-frequency masks from microphone array inputs to achieve user-defined directivity control.
• It utilizes advanced architectures like FT-JNF with bidirectional LSTM and FiLM conditioning, delivering significant SDR gains and low-latency performance in challenging acoustic environments.
• NDF enables on-the-fly reconfigurability of acoustic patterns and robust real-time filtering, outperforming traditional beamforming techniques in diverse applications.

Neural Directional Filtering (NDF) refers to deep neural network-based approaches for spatial audio filtering, enabling single-channel audio capture with programmable directivity patterns using compact microphone arrays. NDF supersedes classical beamforming and parametric spatial filtering techniques by directly inferring time-frequency masks from array inputs, facilitating highly flexible, user-defined pattern control and robust performance in challenging acoustic scenarios.

1. Mathematical Formulation of Directional Filtering

Let $Q$ denote the number of omnidirectional microphones. The short-time Fourier transform (STFT) representation of the signal at microphone $q$ is:

$$Y_q[f,t] = \sum_{n=1}^N H_{q,n}[f] X_n[f,t] + V_q[f,t]$$

Here, $X_n[f,t]$ is the $n$th source signal, $H_{q,n}[f]$ is the acoustic transfer function (ATF) from source $n$ to $q$, and $V_q[f,t]$ is uncorrelated sensor noise. The reference channel is typically $Y_1[f,t]$.

A spatial filter aims to approximate a “virtual directional microphone” (VDM) output, which applies a desired directivity weighting $\Lambda(\theta)$ (azimuth $\theta$) over sources:

$$Z[f,t] = \sum_{k=1}^N \Lambda_t(\theta_k) X_{1,k}[f,t]$$

Classical parametric patterns include differential microphone array (DMA) designs:

$$\Lambda(\theta) = \sum_{j=0}^J a_j \cos^j(\theta - \theta_s)$$

and simplified magnitude-only patterns:

$$\Lambda(\theta;J,\mu,\theta_s) = |\mu + (1-\mu)\cos(\theta - \theta_s)|^J$$

NDF replaces explicit spatial filter calculation with a neural estimator $\mathcal{M}$ generating a complex mask:

$\widehat{Z}[f,t] = M[f,t;\theta_s] \cdot Y_1[f,t]$

Given sufficient training data, $\mathcal{M}$ approximates the target directivity pattern in both amplitude and spatial response.

2. Neural Network Architectures for NDF

The canonical backbone is the FT-JNF (Feature-Transformer Joint Non-linear Filter), typically comprising:

• Input: stacked real/imaginary parts from $Q$ microphones.
• Stages:
  • Bidirectional LSTM across frequency for spectro-spatial modeling.
  • Unidirectional LSTM across time for causal temporal modeling.
• Output head: fully connected layers generating $M[f,t]$.

User-configurable directivity patterns are enabled via conditioning mechanisms:

Pattern Vector JNF (PV-JNF)

The user-supplied pattern vector $p$ (sampled at $L$ directions) is linearly projected to initialize the BiLSTM’s hidden state.

Feature-wise Linear Modulation JNF (FiLM-JNF)

FiLM-based conditioning applies element-wise affine modulation to BiLSTM outputs:

• Compute $\alpha = W_\alpha p + b_\alpha$, $\beta = W_\beta p + b_\beta$.
• FiLM operation: $Y = \alpha \odot X + \beta$.
• $Y$ is then passed through the UniLSTM and mask generation.

This mechanism allows per-frame, per-inference reconfiguration of the directivity pattern vector $p$ with immediate effect.

3. Training Strategies and Pattern Generalization

Training setups use synthesized mixtures of multiple speech sources at randomized directions, direct-path or reverberant RIR simulation, and diverse pattern generation “recipes”:

• Recipe A: First-order DMA patterns over discrete steering angles.
• Recipe B: Random linear combinations of DMA patterns, expanding order and gain variations.
• Recipe B+: Mixture of DMA and rectangular (sector) patterns for approximating highly irregular, non-smooth shapes.

Progressive training refinement proceeds from simple patterns (mainlobe shaping) to complex pattern mixtures (multi-lobe, jagged, simultaneously high-order), enabling the model to generalize to previously unseen pattern geometries. It was empirically shown that FiLM-JNF robustly interpolates to higher directivity orders, scaled lobes, and mixed pattern compositions, outperforming PV-JNF and conventional methods by upwards of 10 dB in SDR for challenging combinations.

4. Performance Analysis and Experimental Evaluation

Comprehensive evaluation utilizes metrics such as:

• SDR (Signal-to-Distortion Ratio): Quantifies fidelity of $\widehat{Z}[f,t]$ to the target VDM output.
• Wideband/Narrowband Directivity Patterns: Derived by applying the neural mask to sources at each test direction $\theta_n$ and computing normalized power ratios.
• Directivity Factor (DF): Ratio of diffuse noise suppression for real-world scenarios.

Representative results with Q=4 microphones in a 3 cm array, anechoic and reverberant conditions:

Method	1st-Order SDR	3rd-Order SDR	6th-Order SDR
LS Beamformer	10.15 dB	–	–
Oracle Parametric	19.27 dB	13.61 dB	10.32 dB
NDF (L₁ loss)	27.30 dB	23.05 dB	18.84 dB

FiLM-JNF with Recipe B+ achieves SDR improvements up to 4 dB over parametric baseline on 3rd-order patterns. Directivity pattern errors converge within ±1 dB for mainlobes and reach –25 dB in nulls; DF gains of the reverberant-trained model can exceed those of theoretical VDMs.

Latency on GPU is approximately 1 ms/frame for the reported architectures. Practical inference for Q≤4, F≤256 is feasible on modern hardware.

5. Low-Latency and Frame-Adaptive NDF

Recent “all-neural” formulations such as the Directional Recurrent Network (DRN) process direction of arrival (DOA) embeddings per frame, fusing spatial and temporal context within recurrent LSTM layers at sub-4 ms increments. Overlap-add output ensures a constant 2 ms algorithmic latency and enables rapid adaptation to abrupt DOA switches within a single frame, outperforming traditional spatial feature engineering in both latency and robustness.

DOA-mismatch experiments show performance degradation is limited (≤0.7 dB SDR within ±10° DOA error).

6. Practical Integration and Application Considerations

• Pattern Reconfigurability: FiLM-based NDF architectures allow on-the-fly reprogramming of filter characteristics, suitable for applications such as automated conference zooming or multi-sector event monitoring.
• Hardware Requirements: The model size and complexity (~0.87 million params) are comparable with conventional neural beamformers; causal design supports real-time deployment.
• Data Requirements: Training with mixtures of multiple sources and densely sampled DOAs is essential for achieving smooth and accurate spatial responses. Including challenging patterns during training is necessary for robust generalization.
• Applications: Use cases include robust speech enhancement, stereo field capture, moving-interferer suppression, and as a neural frontend for downstream ASR or compression systems.

7. Limitations and Future Directions

Current NDF frameworks are mostly evaluated on small (3–9 cm) arrays and in 2D azimuthal domains, with focus on anechoic or low-reverberation conditions. Future research is oriented toward:

• Extending to near-field and fully 3D patterns (elevation and range).
• Generalization to continuous source movement rather than abrupt DOA switches.
• Data-driven adaptation of spatial grids and embedding resolutions.
• Hybridization with classical beamforming for improved robustness to array imperfections or hardware constraints.
• Deployment on ultra-low-power embedded devices and evaluation under real non-simulated acoustic environments.

Neural Directional Filtering establishes a technical foundation for universal, programmable spatial filtering on compact arrays, delivering higher SDR and DF performance than traditional parametric methods and expanding the spectrum of realizable audio directivity patterns in practical systems.