Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?

Abstract: Convolutional networks and vision transformers have different forms of pairwise interactions, pooling across layers and pooling at the end of the network. Does the latter really need to be different? As a by-product of pooling, vision transformers provide spatial attention for free, but this is most often of low quality unless self-supervised, which is not well studied. Is supervision really the problem? In this work, we develop a generic pooling framework and then we formulate a number of existing methods as instantiations. By discussing the properties of each group of methods, we derive SimPool, a simple attention-based pooling mechanism as a replacement of the default one for both convolutional and transformer encoders. We find that, whether supervised or self-supervised, this improves performance on pre-training and downstream tasks and provides attention maps delineating object boundaries in all cases. One could thus call SimPool universal. To our knowledge, we are the first to obtain attention maps in supervised transformers of at least as good quality as self-supervised, without explicit losses or modifying the architecture. Code at: https://github.com/billpsomas/simpool.

• The paper proposes SimPool, a novel pooling mechanism that mitigates attention deficits in supervised transformers.
• It demonstrates a simplified architecture that achieves improved accuracy and efficiency on benchmark tasks.
• The study highlights the practical benefits of reducing model complexity while maintaining high performance in transformer models.

Overview of "LaTeX Author Guidelines for ICCV Proceedings"

The paper "LaTeX Author Guidelines for ICCV Proceedings" serves as a comprehensive guide for authors intending to submit their work to the International Conference on Computer Vision (ICCV). The document prescribes meticulous details about the preparation and formatting of manuscripts using the LaTeX typesetting system, focusing explicitly on adherence to ICCV's standards and submission requirements. This paper does not explore research findings or experimental results; instead, it is a procedural document focused on the technical aspects of manuscript preparation.

The authors detail several critical facets necessary for submission:

1. Language and Dual Submission: It emphasizes the requirement for all manuscripts to be composed in English. There is also guidance regarding ICCV's dual submission policy, encouraging researchers to review the official ICCV guidelines for clarity.
2. Paper Length and Formatting: A strict limit of eight pages for the manuscript, excluding references, is imposed. Overlength papers are not subjected to review, ensuring uniformity and fairness in the evaluation process. Authors must maintain the defined margins and utilize specified font sizes for elements such as titles, headings, and the main text.
3. Reviewer Guidelines: Specific instructions regarding the inclusion of a printed ruler in the submission copy to aid reviewers' commentaries are provided, with additional emphasis on maintaining readability and correct formatting.
4. Mathematics and Equations: It is recommended to number all sections and equations, despite immediate references within the manuscript, to facilitate accessibility for future readers.
5. Blind Review: The document clarifies misunderstandings regarding blind review processes, stressing that while anonymizing author details, referencing one's own work using proper third-person language is acceptable.
6. Illustrations and Figures: Clarity in the inclusion and sizing of figures and illustrations is emphasized to ensure they are comprehensible when printed.
7. Digital Submission Details: The paper contains detailed guidelines for submitting camera-ready copies, including the removal of page numbers and the inclusion of an IEEE copyright release form.

The paper facilitates the seamless creation and submission of conference papers by establishing a standardized formatting practice, significantly reducing the administrative burden on reviewers and editors, and promoting clarity and uniformity across publications. By focusing on technical specifications and practical considerations, this document aids researchers in aligning their work to the rigorous standards of ICCV, thus emphasizing the importance of presentation and precision alongside scientific inquiry.

The implications of these guidelines are clear: they not only enhance the presentation quality but also potentially influence the ease of understanding and impact of the research presented. As the ICCV conference continues to be a pivotal platform for advancements in computer vision, the adherence to these guidelines reinforces the academic rigor and quality of discourse within the community. Future developments may lead to adaptations in the guidelines to accommodate new typesetting technologies or evolving conference standards, maintaining the relevance and applicability of this guide for future ICCV participants.