Omni-Scale CNNs: a simple and effective kernel size configuration for time series classification

Abstract: The Receptive Field (RF) size has been one of the most important factors for One Dimensional Convolutional Neural Networks (1D-CNNs) on time series classification tasks. Large efforts have been taken to choose the appropriate size because it has a huge influence on the performance and differs significantly for each dataset. In this paper, we propose an Omni-Scale block (OS-block) for 1D-CNNs, where the kernel sizes are decided by a simple and universal rule. Particularly, it is a set of kernel sizes that can efficiently cover the best RF size across different datasets via consisting of multiple prime numbers according to the length of the time series. The experiment result shows that models with the OS-block can achieve a similar performance as models with the searched optimal RF size and due to the strong optimal RF size capture ability, simple 1D-CNN models with OS-block achieves the state-of-the-art performance on four time series benchmarks, including both univariate and multivariate data from multiple domains. Comprehensive analysis and discussions shed light on why the OS-block can capture optimal RF sizes across different datasets. Code available [https://github.com/Wensi-Tang/OS-CNN]

• The paper introduces the OS-block, a systematic approach to selecting universal kernel sizes that eliminates the need for dataset-specific tuning.
• It utilizes a prime number-based configuration to achieve state-of-the-art accuracy and improved F1-scores across diverse benchmarks.
• Empirical results demonstrate significant performance gains on medical and general time series datasets, reducing computational costs in model training.

Omni-Scale CNNs: Enhancing Time Series Classification with Universal Kernel Configurations

The paper presents an innovative approach for improving One Dimensional Convolutional Neural Networks (1D-CNNs) in the field of time series classification through the introduction of the Omni-Scale block (OS-block). This contribution is particularly significant as it tackles the critical issue of selecting appropriate kernel sizes for optimizing the receptive field (RF) throughout various datasets, a problem that has traditionally required extensive dataset-specific tuning.

Overview

Time series classification is a challenging task due to the intrinsic variability and complexity of time-dependent data. The selection of RF size in 1D-CNNs is pivotal since it determines the temporal granularity at which features are extracted. Existing methods often resort to exhaustive searches or ad-hoc rules for kernel size determination, which can lead to suboptimal performance or increased computational costs.

The OS-block seeks to address this problem through a systematic method for configuring kernel sizes using a set of prime numbers, which ideally spans a comprehensive range of possible RFs without manual tuning. The kernel set is determined using Goldbach's conjecture as a foundation, ensuring that any RF size up to a pre-defined maximum can be efficiently represented. This theoretically driven yet simple design allows the OS-block to robustly adapt across different datasets by automatically encompassing all potential RF sizes in its configuration.

Strong Numerical Results and Contributions

The paper provides empirical evidence of the superiority of the OS-block through rigorous evaluation on diverse benchmarks including the Magnetoencephalography recording for Temporal Lobe Epilepsy diagnosis dataset and the well-known University of California, Riverside (UCR) archive datasets. The OS-block-equipped 1D-CNNs achieve state-of-the-art results across these benchmarks, affirming their theoretical premise with practical performance gains.

Key results include:

• A noticeable increase in accuracy and F1-score on the Magnetoencephalography dataset over baseline models, with a significant reduction in model size.
• A consistent leading rank in the UEA 30 archive and UCR archives for both univariate and multivariate time series datasets, highlighting the generalizability of the OS-block across different domains and signal complexities.

Implications

The implications of this research are profound for both theoretical exploration and practical deployment. Theoretically, the introduction of a universally applicable kernel size configuration challenges the conventional notion of dataset-specific RF tuning, suggesting that such exhaustive processes might be unnecessary for optimal performance. Practically, the OS-block's automatic configuration could significantly reduce the computational burden associated with model training and hyperparameter tuning, making 1D-CNNs more accessible for diverse applications in fields such as healthcare, financial forecasting, and sensor data analysis.

Future Directions

Looking forward, further exploration of the OS-block could explore enhancing its efficiency even beyond what prime numbers offer, potentially through leveraging machine learning-based tuning during training. Integrating the OS-block with complex neural network architectures like transformers or incorporating attention mechanisms could also yield even greater performance improvements. Moreover, extending the current framework to time series data with varying lengths and irregular sampling would enhance its applicability in real-world scenarios.

In conclusion, the Omni-Scale CNNs present a compelling advancement in time series classification, underpinned by a theoretically sound and universally adaptable kernel configuration strategy. This approach not only achieves strong empirical results but also sets the stage for future innovations in optimizing deep learning models for sequential data analysis.