Papers
Topics
Authors
Recent
Search
2000 character limit reached

Instance-Warp: Saliency Guided Image Warping for Unsupervised Domain Adaptation

Published 19 Mar 2024 in cs.CV and cs.LG | (2403.12712v3)

Abstract: Driving is challenging in conditions like night, rain, and snow. Lack of good labeled datasets has hampered progress in scene understanding under such conditions. Unsupervised Domain Adaptation (UDA) using large labeled clear-day datasets is a promising research direction in such cases. However, many UDA methods are trained with dominant scene backgrounds (e.g., roads, sky, sidewalks) that appear dramatically different across domains. As a result, they struggle to learn effective features of smaller and often sparse foreground objects (e.g., people, vehicles, signs). In this work, we improve UDA training by applying in-place image warping to focus on salient objects. We design instance-level saliency guidance to adaptively oversample object regions and undersample background areas, which reduces adverse effects from background context and enhances backbone feature learning. Our approach improves adaptation across geographies, lighting, and weather conditions, and is agnostic to the task (segmentation, detection), domain adaptation algorithm, saliency guidance, and underlying model architecture. Result highlights include +6.1 mAP50 for BDD100K Clear $\rightarrow$ DENSE Foggy, +3.7 mAP50 for BDD100K Day $\rightarrow$ Night, +3.0 mAP50 for BDD100K Clear $\rightarrow$ Rainy, and +6.3 mIoU for Cityscapes $\rightarrow$ ACDC. Besides, Our method adds minimal training memory and no additional inference latency. Code is available at https://github.com/ShenZheng2000/Instance-Warp

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Accurate image segmentation using gaussian mixture model with saliency map. Pattern Analysis and Applications, 2018.
  2. Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. In CVPR, 2020.
  3. Adascale: Towards real-time video object detection using adaptive scaling. MLSys, 2019.
  4. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016.
  5. Histograms of oriented gradients for human detection. In CVPR, 2005.
  6. Unbiased mean teacher for cross-domain object detection. In CVPR, 2021.
  7. Salisa: Saliency-based input sampling for efficient video object detection. In ECCV, 2022.
  8. Chanakya: Learning runtime decisions for adaptive real-time perception. In NeurIPS, 2023a.
  9. Learned two-plane perspective prior based image resampling for efficient object detection. In CVPR, 2023b.
  10. Deep residual learning for image recognition. In CVPR, 2016.
  11. Cross domain object detection by target-perceived dual branch distillation. In CVPR, 2022.
  12. Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation. In CVPR, 2022a.
  13. HRDA: Context-aware high-resolution domain-adaptive semantic segmentation. In ECCV, 2022b.
  14. MIC: Masked image consistency for context-enhanced domain adaptation. In CVPR, 2023.
  15. Speed/accuracy trade-offs for modern convolutional object detectors. In CVPR, 2017.
  16. Spatial transformer networks. NeurIPS, 2015.
  17. 2pcnet: Two-phase consistency training for day-to-night unsupervised domain adaptive object detection. In CVPR, 2023.
  18. Cross-domain adaptive teacher for object detection. In CVPR, 2022.
  19. Multi-scale context intertwining for semantic segmentation. In ECCV, 2018.
  20. Microsoft coco: Common objects in context. In ECCV, 2014.
  21. Feature pyramid networks for object detection. In CVPR, 2017.
  22. Path aggregation network for instance segmentation. In CVPR, 2018.
  23. David G Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 2004.
  24. Instance adaptive self-training for unsupervised domain adaptation. In ECCV, 2020.
  25. Learning to zoom: a saliency-based sampling layer for neural networks. In ECCV, 2018.
  26. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.
  27. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS, 2015.
  28. Playing for data: Ground truth from computer games. In ECCV, 2016.
  29. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In CVPR, 2016.
  30. Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation. In ICCV, 2019.
  31. Map-guided curriculum domain adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation. TPAMI, 2020.
  32. Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding. In ICCV, 2021.
  33. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.
  34. Training region-based object detectors with online hard example mining. In CVPR, 2016.
  35. An analysis of scale invariance in object detection snip. In CVPR, 2018.
  36. Fovea: Foveated image magnification for autonomous navigation. In ICCV, 2021.
  37. Learning to zoom and unzoom. In CVPR, 2023.
  38. Dacs: Domain adaptation via cross-domain mixed sampling. In WACV, 2021.
  39. Learning to adapt structured output space for semantic segmentation. In CVPR, 2018.
  40. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In CVPR, 2019.
  41. Domain adaptive semantic segmentation with self-supervised depth estimation. In ICCV, 2021.
  42. George Wolberg. Digital image warping. IEEE Computer Society Press Los Alamitos, CA, 1990.
  43. Dannet: A one-stage domain adaptation network for unsupervised nighttime semantic segmentation. In CVPR, 2021.
  44. Segformer: Simple and efficient design for semantic segmentation with transformers. NeurIPS, 2021.
  45. Denseaspp for semantic segmentation in street scenes. In CVPR, 2018.
  46. Fda: Fourier domain adaptation for semantic segmentation. In CVPR, 2020.
  47. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In CVPR, 2020.
  48. Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. In CVPR, 2021.
  49. Scale-transferrable object detection. In CVPR, 2018.
  50. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In ECCV, 2018.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up