Domain Adaptation for Time Series Under Feature and Label Shifts

Abstract: Unsupervised domain adaptation (UDA) enables the transfer of models trained on source domains to unlabeled target domains. However, transferring complex time series models presents challenges due to the dynamic temporal structure variations across domains. This leads to feature shifts in the time and frequency representations. Additionally, the label distributions of tasks in the source and target domains can differ significantly, posing difficulties in addressing label shifts and recognizing labels unique to the target domain. Effectively transferring complex time series models remains a formidable problem. We present Raincoat, the first model for both closed-set and universal domain adaptation on complex time series. Raincoat addresses feature and label shifts by considering both temporal and frequency features, aligning them across domains, and correcting for misalignments to facilitate the detection of private labels. Additionally, Raincoat improves transferability by identifying label shifts in target domains. Our experiments with 5 datasets and 13 state-of-the-art UDA methods demonstrate that Raincoat can improve transfer learning performance by up to 16.33% and can handle both closed-set and universal domain adaptation.

• The paper introduces Raincoat, a novel approach that jointly captures time and frequency domain features to overcome both feature and label shifts.
• It employs Sinkhorn divergence for robust feature alignment, achieving up to 16.33% improvement over state-of-the-art methods.
• The correction step refines target domain adaptation by distinguishing private labels, enabling effective real-world deployment.

Domain Adaptation for Time Series Under Feature and Label Shifts

This paper presents "Raincoat," a novel approach for unsupervised domain adaptation (UDA) in time series data where both feature and label shifts are prevalent. Domain adaptation is essential in real-world applications where models trained on source domain data are expected to perform well on target domain data, which remain unlabeled and possibly drawn from different distributions. This challenge is particularly acute in time series data due to its intrinsic temporal dynamics and potential cross-domain variations in both time and frequency spaces.

Motivation and Challenges

Time series data pose specific challenges for domain adaptation. Unlike static data, time series data are inherently dynamic, leading to feature shifts that manifest in both temporal and frequency domains. Furthermore, label distributions might vary across domains, introducing label shifts that complicate the task further. Previous methods mostly focus on aligning features across domains, often neglecting frequency domain shifts and typically address either feature shift or label shift independently, but not both simultaneously.

Contributions of Raincoat

Raincoat stands out by addressing both closed-set and universal domain adaptation for time series data. It achieves this through several core innovations:

1. Time-Frequency Feature Encoding: Raincoat incorporates a joint time-frequency encoder that captures domain-invariant features. It uses discrete Fourier transform to capture frequency features and combines them with temporal features. This approach leverages the insight that while time-domain signals may exhibit significant domain-specific shifts, frequency domain features can provide complementary, more invariant representations.
2. Sinkhorn Divergence for Feature Alignment: Recognizing the limitations of conventional divergence metrics like MMD and KL divergence in handling disjoint supports, Raincoat employs Sinkhorn divergence for aligning distributions. This metric provides stable gradients even when the supports are disjoint, ensuring robust alignment of complex time series features across domains.
3. Correction Step for Universal Adaptation: To tackle label shifts, Raincoat introduces a correction step that refines the alignment by retraining the encoder on the target domain. By comparing pre- and post-correction embeddings, Raincoat effectively distinguishes private (unknown) labels that are unique to the target domain, facilitating universal domain adaptation.

Empirical Evaluation

Raincoat was evaluated on five diverse datasets encompassing applications from human activity recognition to fault detection and EEG prediction. The results demonstrate substantial improvements over state-of-the-art methods, achieving up to 16.33% better performance in universal domain adaptation scenarios. It outperforms strong baselines, highlighting the effectiveness of incorporating frequency features and a correction step in its adaptation framework.

Theoretical and Practical Implications

The dual focus on both time and frequency domains in Raincoat introduces a new perspective for domain adaptation in time series analytics. The theoretical grounding using Sinkhorn divergence aligns well with the empirical success observed across diverse datasets, showing promise for broader applicability in machine learning scenarios involving complex sequential data.

Practically, Raincoat offers a pathway for deploying models across varied environments without requiring labeled data—an invaluable property in fields like healthcare and autonomous systems where labeling is costly or infeasible.

Future Directions

The paper suggests several avenues for future research. Extending Raincoat to handle other types of data modalities beyond time series can prove valuable. Additionally, investigating more complex architectures, such as transformers for capturing richer temporal dependencies, could further enhance its robustness and applicability. Exploring novel alignment techniques that can automatically balance time and frequency domain contributions based on the context could refine the adaptability of Raincoat.

In conclusion, Raincoat offers a comprehensive solution to domain adaptation challenges in time series data, effectively bridging the gap between source and target domains under complex shifting conditions, and lays the groundwork for future advancements in the field.