Papers
Topics
Authors
Recent
Search
2000 character limit reached

Dual Expert Distillation Network for Generalized Zero-Shot Learning

Published 25 Apr 2024 in cs.CV | (2404.16348v2)

Abstract: Zero-shot learning has consistently yielded remarkable progress via modeling nuanced one-to-one visual-attribute correlation. Existing studies resort to refining a uniform mapping function to align and correlate the sample regions and subattributes, ignoring two crucial issues: 1) the inherent asymmetry of attributes; and 2) the unutilized channel information. This paper addresses these issues by introducing a simple yet effective approach, dubbed Dual Expert Distillation Network (DEDN), where two experts are dedicated to coarse- and fine-grained visual-attribute modeling, respectively. Concretely, one coarse expert, namely cExp, has a complete perceptual scope to coordinate visual-attribute similarity metrics across dimensions, and moreover, another fine expert, namely fExp, consists of multiple specialized subnetworks, each corresponds to an exclusive set of attributes. Two experts cooperatively distill from each other to reach a mutual agreement during training. Meanwhile, we further equip DEDN with a newly designed backbone network, i.e., Dual Attention Network (DAN), which incorporates both region and channel attention information to fully exploit and leverage visual semantic knowledge. Experiments on various benchmark datasets indicate a new state-of-the-art.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (88)
  1. Preserving semantic relations for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7603–7612, 2018.
  2. Synthesized classifiers for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5327–5336, 2016.
  3. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 52–68. Springer, 2016.
  4. Free: Feature refinement for generalized zero-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 122–131, 2021.
  5. Hsva: Hierarchical semantic-visual adaptation for zero-shot learning. Advances in Neural Information Processing Systems, 34:16622–16634, 2021.
  6. Semantics disentangling for generalized zero-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8712–8720, 2021.
  7. Transzero: Attribute-guided transformer for zero-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 330–338, 2022.
  8. Msdn: Mutually semantic distillation network for zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7612–7621, 2022.
  9. Fine-grained side information guided dual-prompts for zero-shot skeleton action recognition. arXiv preprint arXiv:2404.07487, 2024.
  10. Multi-modal cycle-consistent generalized zero-shot learning. In Proceedings of the European Conference on Computer Vision, pages 21–37, 2018.
  11. Adaptive adjustment with semantic feature space for zero-shot recognition. pages 3287–3291, 2019.
  12. Ams-sfe: Towards an alignment of manifold structures via semantic feature expansion for zero-shot learning. pages 73–78, 2019.
  13. Ee-ae: An exclusivity enhanced unsupervised feature learning approach. pages 3517–3521, 2019.
  14. A novel perspective to zero-shot learning: Towards an alignment of manifold structures via semantic feature expansion. IEEE Transactions on Multimedia, 23:524–537, 2020.
  15. Application: Image-based visual perception. pages 123–144, 2022.
  16. Dual-view attention networks for single image super-resolution. pages 2728–2736, 2020.
  17. Conservative novelty synthesizing network for malware recognition in an open-set scenario. IEEE Transactions on Neural Networks and Learning Systems, 2021.
  18. Fed-fsnet: Mitigating non-iid federated learning via fuzzy synthesizing network. arXiv preprint arXiv:2208.12044, 2022.
  19. Cns-net: Conservative novelty synthesizing network for malware recognition in an open-set scenario. arXiv preprint arXiv:2305.01236, 2023.
  20. Graph knows unknowns: Reformulate zero-shot learning as sample-level graph recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7775–7783, 2023.
  21. Sfp: Spurious feature-targeted pruning for out-of-distribution generalization. arXiv preprint arXiv:2305.11615, 2023.
  22. Mdenet: Multi-modal dual-embedding networks for malware open-set recognition. arXiv preprint arXiv:2305.01245, 2023.
  23. Parsnets: A parsimonious orthogonal and low-rank linear networks for zero-shot learning. arXiv preprint arXiv:2312.09709, 2023.
  24. Fine-grained zero-shot learning: Advances, challenges, and prospects. arXiv preprint arXiv:2401.17766, 2024.
  25. Multimodal dual-embedding networks for malware open-set recognition. IEEE Transactions on Neural Networks and Learning Systems, 2024.
  26. Jingcai Guo. An improved incremental training approach for large scaled dataset based on support vector machine. pages 149–157, 2016.
  27. Jingcai Guo. Learning robust visual-semantic mapping for zero-shot learning. arXiv preprint arXiv:2104.05668, 2021.
  28. Contrastive embedding for generalized zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2371–2381, 2021.
  29. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  30. Offline-online class-incremental continual learning via dual-prototype self-augment and refinement. arXiv preprint arXiv:2303.10891, 2023.
  31. Utdnet: A unified triplet decoder network for multimodal salient object detection. Neural Networks, 170:521–534, 2024.
  32. Non-exemplar online class-incremental continual learning via dual-prototype self-augment and refinement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12698–12707, 2024.
  33. Procc: Progressive cross-primitive compatibility for open-world compositional zero-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12689–12697, 2024.
  34. Fine-grained generalized zero-shot learning via dense attribute-based attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4483–4493, 2020.
  35. Transferable contrastive network for generalized zero-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9765–9774, 2019.
  36. Generalized zero-shot learning via over-complete distribution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13300–13308, 2020.
  37. Semantic autoencoder for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3174–3183, 2017.
  38. En-compactness: Self-distillation embedding & contrastive generation for generalized zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9306–9315, 2022.
  39. Learning to detect unseen object classes by between-class attribute transfer. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 951–958. IEEE, 2009.
  40. Leveraging the invariant side of generative zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7402–7411, 2019.
  41. Generalized zero-shot learning via disentangled representation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1966–1974, 2021.
  42. Freepih: Training-free painterly image harmonization with diffusion model. arXiv preprint arXiv:2311.14926, 2023.
  43. Dissecting arbitrary-scale super-resolution capability from pre-trained diffusion generative models. arXiv preprint arXiv:2306.00714, 2023.
  44. Vs-boost: Boosting visual-semantic association for generalized zero-shot learning. In International Joint Conference on Artificial Intelligence, 2023.
  45. Generalized zero-shot learning with deep calibration network. Advances in Neural Information Processing Systems, 31, 2018.
  46. Attribute attention for semantic disambiguation in zero-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6698–6707, 2019.
  47. Attribute propagation network for graph zero-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4868–4875, 2020.
  48. Towards unbiased multi-label zero-shot learning with pyramid and semantic attention. IEEE Transactions on Multimedia, 2022.
  49. Gbe-mlzsl: A group bi-enhancement framework for multi-label zero-shot learning. arXiv preprint arXiv:2309.00923, 2023.
  50. (m⁢l)2superscript𝑚𝑙2(ml)^{2}( italic_m italic_l ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPTp-encoder: On exploration of channel-class correlation for multi-label zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23859–23868, 2023.
  51. Decomposed soft prompt guided fusion enhancing for compositional zero-shot learning. pages 23560–23569, 2023.
  52. Drpt: Disentangled and recurrent prompt tuning for compositional zero-shot learning. arXiv preprint arXiv:2305.01239, 2023.
  53. Position-aware convolutional networks for traffic prediction. arXiv preprint arXiv:1904.06187, 2019.
  54. Latent embedding feedback and discriminative features for zero-shot classification. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16, pages 479–495. Springer, 2020.
  55. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 2751–2758. IEEE, 2012.
  56. Arena: A learning-based synchronization scheme for hierarchical federated learning–technical report. arXiv preprint arXiv:2308.10298, 2023.
  57. Hwamei: A learning-based synchronization scheme for hierarchical federated learning. pages 534–544, 2023.
  58. Improving language understanding by generative pre-training. 2018.
  59. Attribute-aware representation rectification for generalized zero-shot learning. arXiv preprint arXiv:2311.14750, 2023.
  60. Srcd: Semantic reasoning with compound domains for single-domain generalized object detection. arXiv preprint arXiv:2307.01750, 2023.
  61. Generalized zero-and few-shot learning via aligned variational autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8247–8255, 2019.
  62. Personalized federated learning with contextualized generalization. pages 2241–2247, 2022.
  63. Generalized zero-shot learning via synthesized examples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4281–4289, 2018.
  64. Leveraging seen and unseen semantic relationships for generative zero-shot learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, pages 70–86. Springer, 2020.
  65. The caltech-ucsd birds-200-2011 dataset. 2011.
  66. Towards performance-maximizing neural network pruning via global channel attention. Neural Networks.
  67. Dual progressive prototype network for generalized zero-shot learning. Advances in Neural Information Processing Systems, 34:2936–2948, 2021.
  68. Exploring optimal substructure for out-of-distribution generalization via feature-targeted model pruning. arXiv preprint arXiv:2212.09458, 2022.
  69. Efficient stein variational inference for reliable distribution-lossless network pruning. arXiv preprint arXiv:2212.03537, 2022.
  70. Data quality-aware mixed-precision quantization via hybrid reinforcement learning. arXiv preprint arXiv:2302.04453, 2023.
  71. Towards fairer and more efficient federated learning via multidimensional personalized edge models. pages 1–8, 2023.
  72. Towards performance-maximizing neural network pruning via global channel attention. Neural Networks, 171:104–113, 2024.
  73. Zero-shot learning-the good, the bad and the ugly. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4582–4591, 2017.
  74. Feature generating networks for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5542–5551, 2018.
  75. f-vaegan-d2: A feature generating framework for any-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10275–10284, 2019.
  76. Attentive region embedding network for zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9384–9393, 2019.
  77. Region graph embedding network for zero-shot learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pages 562–580. Springer, 2020.
  78. Leveraging balanced semantic embedding for generative zero-shot learning. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  79. Attribute prototype network for zero-shot learning. Advances in Neural Information Processing Systems, 33:21969–21980, 2020.
  80. Episode-based prototype generating network for zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14035–14044, 2020.
  81. Learning a deep embedding model for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2021–2030, 2017.
  82. Towards data-independent knowledge transfer in model-heterogeneous federated learning. IEEE Transactions on Computers, 2023.
  83. Octo:{{\{{INT8}}\}} training with loss-aware compensation and backward quantization for tiny on-device learning. pages 177–191, 2021.
  84. On-device learning systems for edge intelligence: A software and hardware synergy perspective. IEEE Internet of Things Journal, 8(15):11916–11934, 2021.
  85. Cadm: Codec-aware diffusion modeling for neural-enhanced video streaming. arXiv preprint arXiv:2211.08428, 2022.
  86. On the robustness of neural-enhanced video streaming against adversarial attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17123–17131, 2024.
  87. Pass: Patch automatic skip scheme for efficient on-device video perception. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
  88. Semantic-guided multi-attention localization for zero-shot learning. Advances in Neural Information Processing Systems, 32, 2019.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free