PyNeuralFx: A Python Package for Neural Audio Effect Modeling

Abstract: We present PyNeuralFx, an open-source Python toolkit designed for research on neural audio effect modeling. The toolkit provides an intuitive framework and offers a comprehensive suite of features, including standardized implementation of well-established model architectures, loss functions, and easy-to-use visualization tools. As such, it helps promote reproducibility for research on neural audio effect modeling, and enable in-depth performance comparison of different models, offering insight into the behavior and operational characteristics of models through DSP methodology. The toolkit can be found at https://github.com/ytsrt66589/pyneuralfx.

• The paper introduces PyNeuralFx as an open-source toolkit that standardizes neural audio effect modeling for reproducible and comparable research.
• It integrates various model architectures (CNNs and RNNs) and loss functions, enabling robust evaluation with metrics like Loudness Error and Spectral Centroid.
• It offers user-friendly visualization tools for direct waveform comparisons and system response analysis to gain deeper insights into audio processing.

PyNeuralFx: A Python Package for Neural Audio Effect Modeling

The paper presents PyNeuralFx, an open-source Python toolkit engineered to facilitate research in neural audio effect modeling. The toolkit aims to provide a standardized and intuitive framework, complete with a comprehensive suite of features including standardized model architectures, various loss functions, and user-friendly visualization tools. It emphasizes reproducibility and performance comparison, thereby fostering a deeper understanding of neural network-based audio processing systems.

Introduction

Neural audio effect modeling leverages neural networks to emulate a range of audio effects traditionally achieved through digital signal processing (DSP) techniques. While previous studies have showcased the effectiveness of neural networks in achieving high-quality emulation results, variability in training strategies, loss functions, and evaluation metrics across different works has rendered model comparison challenging. In addressing this issue, PyNeuralFx offers a standardized implementation of several training strategies and loss functions, along with visualization tools that facilitate insights into neural networks. The toolkit aims to promote reproducibility and enable more meaningful comparisons between models, helping to advance the field of neural audio effect modeling.

Functionality

Model Architectures

PyNeuralFx implements a diverse range of modeling backbones and control mechanisms. This flexibility allows researchers to explore various neural network structures within a unified framework. The toolkit includes both CNN and RNN based networks:

• CNNs: Temporal Convolutional Networks (TCNs), Gated Convolution Networks (GCNs)
• RNNs: Vanilla RNN, LSTM, GRU

Control mechanisms in the toolkit encompass Concat, FiLM, and several novel conditioning methods for RNNs, such as StaticHyper and DynamicHyper. Notably, PyNeuralFx also introduces a hypernetwork-based conditioning method for CNNs.

Loss Functions for Training

The toolkit provides a comprehensive set of loss functions used in neural audio effect modeling. These include:

• Energy-to-Signal (ESR) Ratio
• Hybrid loss combining Mean Absolute Error and Multi-Resolution Short-Time Fourier Transform Loss
• Short-Time Fourier Transform Complex Loss
• Pre-emphasis filter strategies
• DC Loss for reducing DC offset

These diverse loss functions enhance the toolkit’s versatility and applicability across different modeling scenarios.

Objective Metrics for Evaluation

In addition to the standard reconstruction loss, PyNeuralFx supports several evaluation metrics previously used sporadically, providing a more holistic assessment of model performance. These metrics include:

• Loudness Error
• Crest Factor
• RMS Energy
• Transient Metric
• Spectral Centroid

This comprehensive set of metrics ensures a multifaceted evaluation of models, capturing various dimensions of audio quality and fidelity.

Visualization Tools

To aid in the analysis and interpretation of neural audio effect models, PyNeuralFx offers two types of visualization tools:

• Direct Comparison of Waveforms: Allows visual assessment of similarities and differences between original and processed audio signals.
• System Response Plotting: Observes the model's response to test signals, enabling deeper insights into the network’s behavior.

Usage Flow

The workflow for using PyNeuralFx is streamlined to support ease of experimentation:

1. Dataset Preparation: Researchers can utilize existing datasets like SignalTrain, EGDB, or Boss OD-3, or create custom datasets tailored to specific audio effects.
2. Data Preprocessing: The toolkit provides a standardized template for data preprocessing, ensuring compatibility while maintaining flexibility.
3. Configuration Preparation: Experiments are run based on configuration .yml files, promoting reproducibility.
4. Training, Evaluation, and Visualization: Users can run the training process through PyNeuralFx and evaluate results using various metrics and visualization tools, such as harmonic response and aliasing problem visualizations.

Conclusion

PyNeuralFx represents a significant advancement in the field of neural audio effect modeling, providing a versatile and standardized platform for research. By integrating features from recent studies and offering extensive tools for model evaluation and visualization, PyNeuralFx fosters reproducibility and meaningful model comparison. Future work aims to expand the toolkit’s capabilities with pre-trained models, new architectural advancements, and additional evaluation metrics and loss functions.

The toolkit's repository can be found on GitHub, encouraging community contributions and collaborative research efforts. Through PyNeuralFx, researchers are better equipped to push the boundaries of neural audio effect modeling, driving both practical and theoretical advancements in the field.