Synthesized Binaural Audio Dataset

• Synthesized binaural audio datasets are systematically generated two-channel recordings that simulate spatial sound cues with precise control over azimuth, elevation, and distance.
• They employ methods like HRIR/BRIR convolution, room modeling, and ambisonics-to-binaural decoding to generate high-fidelity, annotated audio signals.
• These datasets enable rigorous benchmarking and support machine learning for source localization, scene analysis, and immersive audio applications.

A synthesized binaural audio dataset is a systematically constructed collection of two-channel audio recordings created by simulating or algorithmically generating spatial sound cues as perceived by the human ear. These datasets are central to research and development in spatial audio, auditory scene analysis, machine learning for source localization, immersive media, and robust human–robot interaction. Unlike datasets based solely on real-world recordings, synthesized binaural corpora allow for precise control over spatial parameters (azimuth, elevation, distance), acoustic conditions, and event composition, while retaining flexibility in scale, diversity, and annotation fidelity.

1. Synthesis Techniques and Core Methodologies

Synthesized binaural datasets employ digital signal processing pipelines to transform isolated monaural source signals into binaural mixtures, generally via convolution with measured or simulated head-related impulse responses (HRIRs) or more complex binaural room impulse responses (BRIRs). The principal methodologies include:

• Direct HRIR/BRIR convolution: Given a monophonic source $s(t)$, binaural signals $x_L(t), x_R(t)$ are produced as $x_L(t) = s(t) * h_L(\theta, \phi)$, $x_R(t) = s(t) * h_R(\theta, \phi)$, where $h_{L/R}$ are HRIRs for the desired direction (Lee et al., 28 Jul 2025). This process supports arbitrary event class insertion—music tracks, speech, or environmental events.
• Room modeling plus HRTF mapping: A geometric simulator generates multi-path room impulse responses (RIRs) using the image source method (ISM), with each path convolved with directionally appropriate HRIRs (Krause et al., 2021, Ratnarajah et al., 2023). This yields rendering that incorporates both direct and reverberant energy, essential for modeling realistic indoor scenes.
• Ambisonics-to-binaural decoding: For datasets originating from first-order ambisonic (FOA) audio, encoded sound fields are spatially rotated and decoded into binaural using time-domain or frequency-domain convolution with HRIRs as a function of camera trajectory in video–audio corpora (Zhang et al., 2 Dec 2025).
• Automated IR interpolation and reverb simulation: Tools like Binamix employ Delaunay triangulation to interpolate measured IRs to arbitrary locations, enabling dense spatial coverage even from sparse measurement grids (Barry et al., 2 May 2025).
• Neural or hybrid approaches: Some frameworks employ generative models (e.g., diffusion models, cGANs) to synthesize binaural signals directly from mono or spatial descriptors, often conditioned on positional metadata, text prompts, or visual information. However, for dataset creation, traditional convolution-based methods remain canonical to provide precise ground truth (Pan et al., 1 Jun 2025, Ratnarajah et al., 2023).

2. Major Datasets: Design, Scale, and Annotation

Synthesized binaural datasets vary widely in structure, content, and intended use. Representative examples include:

Dataset	Source Material & Taxonomy	Spatialization Details	Size / Format
Binaural-MUSDB (Namballa et al., 30 Jun 2025)	MUSDB18-HQ (music, 4 stems)	SADIE II HRIRs (D1), azimuths $-90^\circ$~$+90^\circ$	150 mixtures, ≈10h, 44.1kHz
SpatialTAS (Pan et al., 1 Jun 2025)	AudioSet (>500 events), 1–2 src	HRTFs+room IRs, 8 spatial categories, text prompts	380k clips, ≈293h, 48kHz
BiAudio (Zhang et al., 2 Dec 2025)	Sphere360 FOA+video, open-domain	FOA→binaural (Omnitone HRIRs), camera motion	97k clips, ≈215h, 44.1kHz
Listen2Scene (Ratnarajah et al., 2023)	ScanNet 3D scenes, dry sources	Geometric+material-aware simulation, GNN–cGAN BIRs	1M BIRs, ≈250h, 48kHz
Binaural Set (Lee et al., 28 Jul 2025)	NIGENS, DCASE, 12 events/scene	KAIST HRTF (48 dirs), no reverberation	60,480 scenes, 1,008h, 32kHz
Synthetic Reverberant (Politis et al.) (Krause et al., 2021)	NIGENS, DESED, TUT, 18 events	ISM room modeling + Qu HRIRs (dir/dist dep.)	800 files, ≈3.3h, 24kHz

Synthesized datasets typically annotate each audio file with metadata specifying source event classes, spatial parameters (azimuth, elevation, distance), room properties, and, when applicable, text prompts, camera trajectories, or per-frame active class indicators.

3. Spatial Rendering and Parameterization

Spatial rendering in synthesized binaural datasets is governed by:

• HRIR/BRIR libraries: Usage of high-resolution, multi-subject measured datasets (e.g., SADIE II (Barry et al., 2 May 2025), KAIST (Lee et al., 28 Jul 2025), Qu et al. (Krause et al., 2021)) enables sampling of arbitrary or densely interpolated source directions.
• Angle and distance grids: Some corpora restrict to fixed azimuth–elevation grids (e.g., Binaural Set: 12 azimuths × 4 elevations) for uniform coverage and tractable training/evaluation (Lee et al., 28 Jul 2025).
• Room and scene modeling: Detailed randomization of room geometry (size, RT60), source–receiver positions, and environmental class (material, topology) is applied in room-aware datasets (Krause et al., 2021, Ratnarajah et al., 2023). This supports both direction-of-arrival (DoA) and proximity estimation.
• Annotation protocols: Metadata may include per-event spatial labels (θ, φ, d), event class, precise onset/offset times, and, in complex cases, relative spatial relations or text natural language prompts for interactive experiments (Pan et al., 1 Jun 2025).
• Dynamic/viewpoint adaptation: Video-driven pipelines (e.g., BiAudio (Zhang et al., 2 Dec 2025)) parameterize spatialization to match temporally varying camera pose, ensuring spatial audio remains consistent with visual context.

4. Tools and Reproducible Pipelines

The proliferation of open-source toolkits has substantially lowered the barrier to custom dataset generation:

• Binamix (Barry et al., 2 May 2025): Provides Python APIs for IR reading, angle interpolation using Delaunay triangulation, multi-track mixing with configurable HRIR/BRIR sources, and batch export. Users script over subjects, azimuth/elevation combinations, and reverb profiles to systematically cover large condition spaces.
• Shoebox-roomsim (Krause et al., 2021): Employed for ISM-based RIR generation in synthetic room scenarios, in combination with directionally aligned HRIRs.
• Ambisonic decoders: For datasets starting from ambisonics, standard libraries (e.g., Omnitone (Zhang et al., 2 Dec 2025)) are used for FOA-to-binaural conversion, possibly augmented by scene-driven spatial filtering.

Best practices emerging from these pipelines include output-level normalization, careful anti-clipping safeguards, and the use of high-bit-depth output formats for downstream audio analysis (Barry et al., 2 May 2025).

5. Evaluation, Validation, and Use Cases

Synthesized binaural datasets are evaluated along several axes:

• Objective metrics: Signal-based localization error (ITD/ILD accuracy), SELD (sound event localization and detection) error, direction/proximity classification accuracy, reverberation time (T60), direct-to-reverberant ratio (DRR), and energy-decay curve (EDC) MSE. For Listen2Scene, improvement over baselines in T60, DRR, and early decay time (EDT) is reported (Ratnarajah et al., 2023).
• Subjective evaluation: Perceptual tests (AB comparison, MOS) validate the plausibility and spatial fidelity of synthetic audio versus real data. For example, Listen2Scene observed a 71% preference rate over “clean” non-material-aware rendering (Ratnarajah et al., 2023).
• Semantic tasks: Large-scale datasets such as SpatialTAS enable text-guided or multimodal experiments (e.g., conditioning generation on natural language spatial prompts), supporting research into spatial semantic coherence (Pan et al., 1 Jun 2025).
• Machine learning training: Datasets with exact spatial ground truth are heavily exploited for supervised learning of localization models, CRNN-based classification, and, increasingly, generative models for spatial audio synthesis and separation (Lee et al., 28 Jul 2025, Namballa et al., 30 Jun 2025).

6. Strengths, Limitations, and Research Applications

The principal advantage of synthesized binaural audio datasets is the precise, programmatic control of spatial attributes, scene complexity, and annotation. This is particularly beneficial in:

• Controlled benchmarking: Enabling rigorous SELD evaluation and ablation analysis, with perfect ground-truth event–space mappings (Lee et al., 28 Jul 2025, Krause et al., 2021).
• Generalization studies: Large-scale synthetic data are used for domain adaptation, zero-shot synthesis, and cross-condition testing of models, with explicit test sets for in-distribution and out-of-distribution evaluation (Namballa et al., 30 Jun 2025).

However, such datasets are limited by:

• Lack of real-environment artifacts: Simulations may not capture fine-scale reverberation, motion, or individual anatomical HRTF variation (Pan et al., 1 Jun 2025, Zhang et al., 2 Dec 2025).
• Simplified dynamics: Many corpora fix all sources as static within the duration of each clip; dynamic role, occlusion, and certain psychoacoustic cues are typically omitted.

Despite these gaps, synthesized binaural datasets are essential for scalable research in immersive audio, machine hearing, and multimodal spatial scene understanding. They underpin state-of-the-art source separation, text–audio spatialization, robotic audition, and video-driven sound synthesis (Namballa et al., 30 Jun 2025, Zhang et al., 2 Dec 2025, Lee et al., 28 Jul 2025).

7. Access, Extension, and Community Resources

Most major synthesized binaural datasets and supporting code are released openly for academic use. Distribution is typically via public repositories or web portals, with standardized directory structures for audio and annotation files, and clear file-naming conventions for split, index, and parameter encoding.

• Binamix: https://github.com/QxLabIreland/Binamix/ (Barry et al., 2 May 2025)
• SpatialTAS: https://github.com/Alice01010101/TASU (Pan et al., 1 Jun 2025)
• Listen2Scene: https://anton-jeran.github.io/Listen2Scene/ (Ratnarajah et al., 2023)
• Binaural Set: (Release implied in (Lee et al., 28 Jul 2025))

A plausible implication is that the continued synthesis of ever-larger, more realistic, and richly annotated binaural datasets will remain vital to progress in spatial audio modeling, benchmarking, and cross-modal learning systems.