Loudness Encoder Overview

• Loudness encoder is a computational module that simulates human auditory perception using psychoacoustic models and filterbank techniques to quantify sound intensity in phons or sones.
• Recent designs employ deep neural networks and time-domain filterbanks to deliver high-accuracy, real-time loudness estimations across audio, speech, music, and cochlear implant applications.
• Diverse methodologies, from classical psychoacoustic algorithms to ML-driven models, balance computational efficiency, fidelity to human perception, and integration with advanced audio processing systems.

A loudness encoder is a computational module or algorithm that estimates perceived loudness (typically in phons or sones) from sensory inputs, such as audio waveforms or electrical pulse trains, by simulating human auditory perception. Distinct designs have been proposed for acoustic audio, speech, music, and cochlear implant scenarios, often balancing fidelity to psychophysical standards, computational efficiency, and compatibility with larger machine-learning systems.

1. Psychoacoustic Foundations and Standard Models

Classic approaches to loudness encoding are anchored in psychoacoustic theory, where perceived loudness is determined via models reflecting the nonlinear, frequency-dependent mapping from physical stimulus properties to sensation magnitude. The Moore–Glasberg (ISO 532-2) model and its predecessors specify computational procedures for broadband sounds, involving:

• Outer/middle-ear filtering,
• Critical-band decomposition (using filterbanks such as roex or ERB-spaced gammatone/gammachirp channels),
• Nonlinear transduction (applying frequency-dependent thresholds and compressive gain functions),
• Across-band aggregation yielding a total loudness metric in sones or phons.

Recent revisions in ISO 532 recognize both Moore–Glasberg (ISO 532-2) and Zwicker (ISO 532-1) methods as standards. These methods form the reference targets or baselines for most advanced loudness encoders (Isoyama et al., 2023).

2. Deep Neural-Network Loudness Encoders

Schlittenlacher et al. introduced a deep neural-network (DNN) loudness encoder that simulates the Cambridge loudness model (ISO 532-2/3) but operates orders of magnitude faster (Schlittenlacher et al., 2019). The architecture is a multilayer perceptron (MLP) with the following characteristics:

• Input representation: The raw audio is segmented into 35 ms frames (560 samples at 16 or 44.1 kHz), zero-padded, transformed via a 1024-point DFT, and binned into 61 frequency bands (constant bandwidth up to 200 Hz, then nine bands per octave up to 8 kHz), producing a 61-dimensional vector in dB SPL.
• Architecture: Three hidden layers (each 150 ReLU units), followed by a scalar linear output. The full forward mapping is:

$$L_{\mathrm{pred}}(x) = W^4\,\text{ReLU}(W^3\,\text{ReLU}(W^2\,\text{ReLU}(W^1 x + b^1) + b^2) + b^3) + b^4$$

where $x\in\mathbb{R}^{61}$.

• Training: Targets are instantaneous loudness values from the Cambridge model. The network minimizes mean squared error (MSE) on roughly 1.7M diverse frames (speech, tones in noise, band-limited/noisy spectra). Adam optimizer is employed across three training phases (total ≈5000 epochs).
• Performance: RMS deviation from the Cambridge model is <0.5 phon on all test sets, below the just-noticeable difference for human listeners. Computation rates exceed 100,000 predictions/sec, >100× faster than the original reference implementation, thus enabling real-time use.

3. Time-Domain Auditory Filterbank Encoders

A distinct class of loudness encoders replaces frequency-domain filterbanks with real-time, time-domain implementations—specifically, gammatone (GT) and gammachirp (GC) filterbanks (Isoyama et al., 2023). These models operate as follows:

• Processing chain:

1. Outer/middle-ear FIR filtering.
2. Time-domain filtering of the prefiltered signal via cascaded GT or GC IIR sections across 344–372 channels, spaced by equivalent rectangular bandwidth (ERB-number).
3. Envelope extraction per band through half-wave rectification, squaring, and double first-order low-pass filtering (cutoff 1200 Hz).
4. Specific loudness per band computed using ISO 532-2 nonlinearities and thresholds.
5. Summation across all channels yields instantaneous loudness $N(t)$ in sones.

• Parameterization: Channel center frequencies span ERB-number 1.8–38.9 (GT) or 2.6–36.9 (GC), with precise formulas for filter shape and loudness calculation detailed in LaTeX notation.
• Validation: The GT- and GC-based loudness encoders closely match the Moore–Glasberg reference, with RMSE <0.2 sones (2–3% of typical loudness values) across various pure-tone stimuli—allowing integration into computational frameworks for sound-quality metrics.

4. Loudness Encoding in Neuroprosthetic (Cochlear Implant) Systems

Novel loudness encoders for cochlear implant (CI) applications have been designed to model the neural population dynamics underlying loudness perception for electrical stimulation (Alvarez et al., 29 Jan 2025):

• Peripheral modeling: A detailed 3D finite-element solution of the cochlea and an implanted 22-electrode array yields the extracellular voltage distribution for each electrode and fiber.
• Neural simulation: Each of 30,000 simulated ANFs is modeled using a dual-exponential integrate-and-fire mechanism, receiving stimulation as proportional to local field maxima. Spikes are recorded and aggregated within 40 contiguous fiber groups (representing auditory filter channels).
• Loudness computation: Per-channel spike rates are transformed by a two-stage nonlinearity (linear gain and exponential spectral summation). Temporal integration applies an asymmetric window simulating psychoacoustic temporal summation/masking. Lateral summation across channels forms $\ell(t)$, and the loudness index $L = \mathrm{percentile}_{99}\{\ell(t)\}$.
• Calibration and validation: Threshold (THL) and most comfortable loudness (MCL) are set at $L=5$ and $L=100$, respectively. The model reproduces current/rate and spatial summation effects observed in human loudness psychophysics for CI users, providing a physiologically grounded method for loudness estimation in CI fitting and research.

5. Machine-Learning Loudness Encoders for Music and TTS

Data-driven loudness encoders have been applied in both music and speech processing contexts, focusing on efficient representations for downstream control or conditioning.

5.1. Piano Tone Loudness (Pairwise ML)

In piano performance transfer, perceptual loudness is modeled using empirically measured equal-loudness contours and a linear ML model (Qu et al., 2022):

• Measurement: Point of subjective equality in loudness across pitch and dynamic is determined through forced-choice psychophysical experiments, forming ELCs (Equal-Loudness Contours).
• Model: Supervised pairwise ranking on short (0.1 s) 8×5-bin mel-spectrogram features, with a linear regression output calibrated to yield sones (ISO 532-3 alignment).
• Application: Used to match perceived loudness across different instruments and environments by solving for velocity remapping that minimizes loudness deviation.
• Empirical performance: Outperforms intensity-based models; achieves near-perfect pairwise prediction and improved perceptual transfer in listening tests.

5.2. Loudness Encoder in Neural TTS

In prosody transfer for neural text-to-speech, loudness is encapsulated as global statistics, eschewing frame-wise dynamics for computational parsimony (Gururani et al., 2019):

• Feature extraction: RMS energy is computed over 50 ms frames (12.5 ms hop), without decibel conversion.
• Encoding: Mean, variance, and max of framewise RMS across an utterance are concatenated with global pitch features, mean-variance normalized, and projected via a 7×512 linear layer to yield a prosody embedding.
• Usage: This embedding is added to every encoder state in Tacotron2, conditioning the entire TTS model.
• Training: No auxiliary loss is introduced; the encoder relies on tacitly learning to use the embedding for prosody shaping.
• Evaluation: Global-statistics prosody encoding reduces RMS cosine distance to reference (0.027 vs 0.034, lower is better) and is preferred in subjective prosody transfer comparisons.

6. Comparative Analysis and Implementation Summary

The table below summarizes representative loudness encoder types, input modalities, and key properties:

Encoder Type	Input and Features	Output/Performance
DNN MLP (Cambridge approx)	61-bin log-DFT, 35 ms frames	$<$0.5 phon RMS error, 100,000+ Hz
Time-domain GT/GC filter	Pre-filtered waveform, 344–372 bands	$<$0.2 sones RMSE, real-time
CI neural population model	3D ENI, 30k ANFs, pulse train	Validated on CI user psychophys.
ML piano pairwise model	8×5 mel, 0.1 s post-attack	Accurate sones, music transfer
Global-stats TTS encoder	Framewise RMS, global mean/var/max	Improved prosody transfer

Each design reflects the specific constraints of its application—real-time loudness tracking in large datasets, compatibility with auditory neuroprostheses, or efficient and disentangled prosody control in generative models.

7. Practical Considerations, Limitations, and Extensions

Identified limitations include band-limited noise underestimation in MLPs that lack explicit across-frequency integration (Schlittenlacher et al., 2019); potential sensitivity to adversarial spectra; and dependency on the choice and granularity of statistical features in global-statistics encoders (Gururani et al., 2019). In CI models, full 3D field computation and population IF-models increase implementation complexity, though they offer direct physiological interpretability (Alvarez et al., 29 Jan 2025). For music and TTS, statistical or linear projections provide interpretable, data-efficient embeddings but may compromise fine-grained temporal fidelity.

Extensions proposed include adapting fast DNN surrogates for other perceptual models (intelligibility indices, spatial hearing), enriching context with CNN/RNN temporal encoders, or integrating physiologically-informed models with real-time audio devices. Empirical validation against psychoacoustic thresholds and human subject experiments remains the gold standard for deployment readiness across all loudness encoder paradigms.