2000 character limit reached
Event-assisted Low-Light Video Object Segmentation
Published 2 Apr 2024 in cs.CV | (2404.01945v1)
Abstract: In the realm of video object segmentation (VOS), the challenge of operating under low-light conditions persists, resulting in notably degraded image quality and compromised accuracy when comparing query and memory frames for similarity computation. Event cameras, characterized by their high dynamic range and ability to capture motion information of objects, offer promise in enhancing object visibility and aiding VOS methods under such low-light conditions. This paper introduces a pioneering framework tailored for low-light VOS, leveraging event camera data to elevate segmentation accuracy. Our approach hinges on two pivotal components: the Adaptive Cross-Modal Fusion (ACMF) module, aimed at extracting pertinent features while fusing image and event modalities to mitigate noise interference, and the Event-Guided Memory Matching (EGMM) module, designed to rectify the issue of inaccurate matching prevalent in low-light settings. Additionally, we present the creation of a synthetic LLE-DAVIS dataset and the curation of a real-world LLE-VOS dataset, encompassing frames and events. Experimental evaluations corroborate the efficacy of our method across both datasets, affirming its effectiveness in low-light scenarios.
- Ev-segnet: Semantic segmentation for event-based cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
- A 240 ×\times× 180 130 db 3 μ𝜇\muitalic_μs latency global shutter spatiotemporal vision sensor. IEEE Journal of Solid-State Circuits, 49(10):2333–2341, 2014.
- One-shot video object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 221–230, 2017.
- State-aware tracker for real-time video object segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9384–9393, 2020.
- Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model. In European Conference on Computer Vision (ECCV), pages 640–658. Springer, 2022.
- Rethinking space-time networks with improved memory coverage for efficient video object segmentation. Advances in Neural Information Processing Systems, 34:11781–11794, 2021.
- Video to events: Recycling video datasets for event cameras. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2020.
- Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 1780–1789, 2020.
- Video propagation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 451–461, 2017.
- Event-based low-illumination image enhancement. IEEE Transactions on Multimedia, pages 1–12, 2023a.
- Event-based low-illumination image enhancement. IEEE Transactions on Multimedia, 26:1920–1931, 2023b.
- Recurrent dynamic embedding for video object segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1332–1341, 2022.
- Coherent event guided low-light video enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10615–10625, 2023.
- Low-light video enhancement with synthetic event guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1692–1700, 2023.
- Attention guided low-light image enhancement with a large scale low-light simulation dataset. International Journal of Computer Vision, 129(7):2175–2193, 2021.
- Joint inductive and transductive learning for video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9670–9679, 2021.
- Learning visual motion segmentation using event surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14414–14423, 2020.
- Fast video object segmentation by reference-guided mask propagation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7376–7385, 2018.
- Video object segmentation using space-time memory networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019a.
- Video object segmentation using space-time memory networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9226–9235, 2019b.
- Learning video object segmentation from static images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2663–2672, 2017.
- Film: Frame interpolation for large motion. In European Conference on Computer Vision, pages 250–266. Springer, 2022.
- Kernelized memory network for video object segmentation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16, pages 629–645. Springer, 2020.
- Hierarchical memory matching network for video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12889–12898, 2021.
- Even: An event-based framework for monocular depth estimation at adverse night conditions. arXiv preprint arXiv:2302.03860, 2023.
- Event-based motion segmentation by motion compensation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7244–7253, 2019.
- Ess: Learning event-based semantic segmentation from still images. In European Conference on Computer Vision (ECCV), pages 341–357. Springer, 2022.
- Front and back illuminated dynamic and active pixel vision sensors comparison. IEEE Transactions on Circuits and Systems II: Express Briefs, 65(5):677–681, 2018.
- Selective video object cutout. IEEE Transactions on Image Processing, 26(12):5645–5655, 2017.
- Cmda: Cross-modality domain adaptation for nighttime semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 21572–21581, 2023.
- Monet: Deep motion exploitation for video object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1140–1148, 2018.
- Reliable propagation-correction modulation for video object segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2946–2954, 2022.
- Decoupling features in hierarchical propagation for video object segmentation. Advances in Neural Information Processing Systems, 35:36324–36336, 2022.
- Associating objects with transformers for video object segmentation. Advances in Neural Information Processing Systems, 34:2491–2502, 2021.
- Fast video object segmentation via dynamic targeting network. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5582–5591, 2019.
- Learning to see in the dark with events. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, pages 666–682. Springer, 2020.
- Instance-level segmentation for autonomous driving with deep densely connected mrfs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 669–677, 2016.
- Delieve-net: Deblurring low-light images with light streaks and local events. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1155–1164, 2021a.
- Deblurring low-light images with events. International Journal of Computer Vision, 131(5):1284–1298, 2023.
- Event-based motion segmentation with spatio-temporal graph cuts. IEEE Transactions on Neural Networks and Learning Systems, 34(8):4868–4880, 2021b.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.