Correlation Alignment for Unsupervised Domain Adaptation

Abstract: In this chapter, we present CORrelation ALignment (CORAL), a simple yet effective method for unsupervised domain adaptation. CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any target labels. In contrast to subspace manifold methods, it aligns the original feature distributions of the source and target domains, rather than the bases of lower-dimensional subspaces. It is also much simpler than other distribution matching methods. CORAL performs remarkably well in extensive evaluations on standard benchmark datasets. We first describe a solution that applies a linear transformation to source features to align them with target features before classifier training. For linear classifiers, we propose to equivalently apply CORAL to the classifier weights, leading to added efficiency when the number of classifiers is small but the number and dimensionality of target examples are very high. The resulting CORAL Linear Discriminant Analysis (CORAL-LDA) outperforms LDA by a large margin on standard domain adaptation benchmarks. Finally, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (DNNs). The resulting Deep CORAL approach works seamlessly with DNNs and achieves state-of-the-art performance on standard benchmark datasets. Our code is available at:~\url{https://github.com/VisionLearningGroup/CORAL}

• The paper introduces CORAL, a method that aligns source and target covariance matrices to mitigate domain shift in unsupervised learning.
• It employs a linear transformation by whitening and re-coloring features, offering a simpler alternative to traditional manifold projection methods.
• Empirical results in object detection and deep network adaptation demonstrate CORAL’s effectiveness, especially when integrated with LDA and deep learning frameworks.

Unsupervised Domain Adaptation Using Correlation Alignment

The paper "Correlation Alignment for Unsupervised Domain Adaptation" introduces CORrelation ALignment (CORAL), a technique designed to mitigate domain shift in unsupervised learning by aligning the second-order statistics of source and target feature distributions. Unlike subspace manifold methods, CORAL directly aligns the original feature distributions, simplifies implementation, and exhibits superior performance in multiple benchmark evaluations.

Linear CORAL Methodology

Derivation and Implementation

CORAL employs a linear transformation to align covariance matrices of source and target domains, minimizing the Frobenius norm of their difference. The transformation involves whitening source features and re-coloring them with the target covariance. This linear solution avoids complex subspace projections, making it both efficient and easy to implement. The algorithm can be summarized in a few lines of MATLAB, illustrating its simplicity relative to traditional domain adaptation techniques involving hyperparameter tuning and dimensionality selection.

Comparative Analysis

The feature normalization in CORAL goes beyond conventional batch normalization techniques by actively realigning feature covariances, which batch normalization fails to address in domain-shift scenarios. It also diverges from Maximum Mean Discrepancy (MMD) approaches that attempt symmetric transformations on both domains. CORAL's asymmetric transformation, which leverages the distinct characteristics of source and target data, consistently outperforms manifold alignment methods by integrating both eigenvectors and eigenvalues for a comprehensive distribution alignment.

CORAL-LDA for Object Detection

Integration with Linear Discriminant Analysis

By incorporating the CORAL approach into Linear Discriminant Analysis (LDA), the paper proposes CORAL-LDA, a robust method for object detection that decorrelates and aligns feature distributions across domains. This is vital in scenarios where virtual data serve as training inputs aimed at recognizing real-world images. The covariance adjustments derived from target domain statistics significantly enhance the detection accuracy over models relying solely on source domain statistics or even large-scale unrelated datasets like PASCAL.

Performance and Limitations

Experiments demonstrate that CORAL-LDA achieves high Mean Average Precision (MAP) values in detection tasks across varying source domains and proves its efficacy in unsupervised adaptation using limited labeled data from the target domain. The approach also accommodates semi-supervised settings where a small number of labeled target samples further heightens accuracy. However, reliance on covariance alignment alone may disregard higher-order data structures, yet the empirical results validate the substantial gains over traditional methods in diverse adaptation tasks.

Deep CORAL for Nonlinear Representations

Extending CORAL with Deep Learning

The development of Deep CORAL extends the alignment paradigm to deep neural networks, introducing a differentiable layer that minimizes correlation distance within network activations. This integration supports end-to-end adaptation, facilitating fine-tuning across multiple network depths. The architecture of Deep CORAL leverages a joint optimization of classification and CORAL losses, achieving a balance that preserves discriminative power while ensuring domain-invariant features.

Experimental Results and Insights

Applying Deep CORAL to standard object recognition datasets shows improvements over several baseline methods, including DDC and DAN. The adaptation equilibriums reached by Deep CORAL indicate a stable convergence, enhancing target domain prediction accuracy without compromising source domain learnings.

Conclusion

The CORAL methodology presents a pragmatic solution for unsupervised domain adaptation by tackling the intrinsic challenges of domain-shift through covariance alignment. Although primarily targeting second-order statistics, CORAL's practical deployment across linear and deep learning frameworks demonstrates broad applicability and effectiveness in tasks ranging from object recognition to detection in cross-domain scenarios. Future exploration could explore integration with higher-order feature alignments for comprehensive domain adaptation solutions.