Papers
Topics
Authors
Recent
Search
2000 character limit reached

Group-On: Boosting One-Shot Segmentation with Supportive Query

Published 18 Apr 2024 in cs.CV | (2404.11871v2)

Abstract: One-shot semantic segmentation aims to segment query images given only ONE annotated support image of the same class. This task is challenging because target objects in the support and query images can be largely different in appearance and pose (i.e., intra-class variation). Prior works suggested that incorporating more annotated support images in few-shot settings boosts performances but increases costs due to additional manual labeling. In this paper, we propose a novel and effective approach for ONE-shot semantic segmentation, called Group-On, which packs multiple query images in batches for the benefit of mutual knowledge support within the same category. Specifically, after coarse segmentation masks of the batch of queries are predicted, query-mask pairs act as pseudo support data to enhance mask predictions mutually. To effectively steer such process, we construct an innovative MoME module, where a flexible number of mask experts are guided by a scene-driven router and work together to make comprehensive decisions, fully promoting mutual benefits of queries. Comprehensive experiments on three standard benchmarks show that, in the ONE-shot setting, Group-On significantly outperforms previous works by considerable margins. With only one annotated support image, Group-On can be even competitive with the counterparts using 5 annotated images.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Integrative few-shot learning for classification and segmentation. In Conference on Computer Vision and Pattern Recognition, 2022.
  2. BriNet: Towards bridging the intra-class and inter-class gaps in one-shot segmentation. arXiv preprint arXiv:2008.06226, 2020.
  3. Suppressing the heterogeneity: A strong feature extractor for few-shot segmentation. In International Conference on Learning Representations, 2023.
  4. FECANet: Boosting few-shot semantic segmentation with feature-enhanced context-aware network. arXiv preprint arXiv:2301.08160, 2023.
  5. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88:303–338, 2010.
  6. Microsoft COCO: Common objects in context. In European Conference on Computer Vision, 2014.
  7. FSS-1000: A 1000-class dataset for few-shot segmentation. In Conference on Computer Vision and Pattern Recognition, 2020.
  8. One-shot learning for semantic segmentation. In British Machine Vision Conference, 2017.
  9. Dense Gaussian processes for few-shot segmentation. In European Conference on Computer Vision, 2022.
  10. Rethinking the correlation in few-shot segmentation: A buoys view. In Conference on Computer Vision and Pattern Recognition, 2023.
  11. MSANet: Multi-similarity and attention guidance for boosting few-shot segmentation. arXiv preprint arXiv:2206.09667, 2022.
  12. Elimination of non-novel segments at multi-scale for few-shot segmentation. In Winter Conference on Applications of Computer Vision, 2023.
  13. Few-shot segmentation via cycle-consistent Transformer. arXiv preprint arXiv:2106.02320, 2022.
  14. Dense cross-query-and-support attention weighted mask aggregation for few-shot segmentation. In European Conference on Computer Vision, 2022.
  15. Hierarchical dense correlation distillation for few-shot segmentation. arXiv preprint arXiv:2303.14652, 2023.
  16. Universal few-shot learning of dense prediction tasks with visual token matching. In International Conference on Learning Representations, 2023.
  17. CATrans: Context and affinity Transformer for few-shot segmentation. In International Joint Conferences on Artificial Intelligence, 2022.
  18. Intermediate prototype mining Transformer for few-shot semantic segmentation. In Advances in Neural Information Processing Systems, 2022.
  19. Adaptive agent transformer for few-shot segmentation. In European Conference on Computer Vision, 2022.
  20. Hypercorrelation squeeze for few-shot segmentation. In International Conference on Computer Vision, 2021.
  21. Cost aggregation with 4D convolutional Swin Transformer for few-shot segmentation. In European Conference on Computer Vision, 2022.
  22. Doubly deformable aggregation of covariance matrices for few-shot segmentation. In European Conference on Computer Vision, 2022.
  23. PANet: Few-shot image semantic segmentation with prototype alignment. In International Conference on Computer Vision, 2019.
  24. Prototype mixture models for few-shot semantic segmentation. In European Conference on Computer Vision, 2020.
  25. Mining latent classes for few-shot segmentation. In International Conference on Computer Vision, 2021.
  26. Learning what not to segment: A new perspective on few-shot segmentation. In Conference on Computer Vision and Pattern Recognition, 2022.
  27. HM: Hybrid masking for few-shot segmentation. In European Conference on Computer Vision, 2022.
  28. MSI: Maximize support-set information for few-shot segmentation. arXiv preprint arXiv:2212.04673, 2023.
  29. Atsuro Okazawa. Interclass prototype relation for few-shot segmentation. In European Conference on Computer Vision, 2022.
  30. MIANet: Aggregating unbiased instance and general information for few-shot semantic segmentation. In Conference on Computer Vision and Pattern Recognition, 2023.
  31. Learning from the target: Dual prototype network for few shot semantic segmentation. In AAAI Conference on Artificial Intelligence, 2022.
  32. Self-support few-shot semantic segmentation. In European Conference on Computer Vision, 2022.
  33. Few-shot segmentation with global and local contrastive learning. arXiv preprint arXiv:2108.05293, 2021.
  34. Optimization as a model for few-shot learning. In International Conference on Learning Representations, 2017.
  35. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, 2016.
  36. SG-One: Similarity guidance network for one-shot semantic segmentation. IEEE Transactions on Cybernetics, 50(9):3855–3865, 2020.
  37. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition, 2016.
  38. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
  39. Quaternion-Valued correlation learning for few-shot semantic segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 33(5):2102–2115, 2023.
  40. Masked cross-image encoding for few-shot segmentation. arXiv preprint arXiv:2308.11201, 2023.
  41. Self-calibrated cross attention network for few-shot segmentation. In International Conference on Computer Vision, 2023.
  42. Self-supervised few-shot learning for semantic segmentation: An annotation-free approach. In Medical Image Computing and Computer Assisted Intervention, 2023.
  43. DifFSS: Diffusion model for few-shot semantic segmentation. arXiv preprint arXiv:2307.00773, 2023.
  44. Prior guided feature enrichment network for few-shot segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(2):1050–1065, 2022.
  45. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020, 2021.
  46. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, 2019.
  47. ImageNet: A large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition, 2009.
  48. Deep sparse rectifier neural networks. In International Conference on Artificial Intelligence and Statistics, 2011.
  49. Karl Pearson. Notes on regression and inheritance in the case of two parents. In Royal Society of London, 1895.
  50. C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101, 1904.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free