Are Labels Necessary for Neural Architecture Search?

Abstract: Existing neural network architectures in computer vision -- whether designed by humans or by machines -- were typically found using both images and their associated labels. In this paper, we ask the question: can we find high-quality neural architectures using only images, but no human-annotated labels? To answer this question, we first define a new setup called Unsupervised Neural Architecture Search (UnNAS). We then conduct two sets of experiments. In sample-based experiments, we train a large number (500) of diverse architectures with either supervised or unsupervised objectives, and find that the architecture rankings produced with and without labels are highly correlated. In search-based experiments, we run a well-established NAS algorithm (DARTS) using various unsupervised objectives, and report that the architectures searched without labels can be competitive to their counterparts searched with labels. Together, these results reveal the potentially surprising finding that labels are not necessary, and the image statistics alone may be sufficient to identify good neural architectures.

• The paper presents UnNAS, demonstrating that effective neural architectures can be identified without relying on labeled data.
• It employs both sample-based experiments and a modified DARTS approach, revealing a high rank correlation between unsupervised and supervised performance.
• Results indicate that leveraging large unlabeled datasets may outperform traditional methods, reducing the need for costly manual annotations.

An Examination of Label Independence in Neural Architecture Search

In the study entitled "Are Labels Necessary for Neural Architecture Search?", the authors explore a provocative question within computer vision: can effective neural architectures be identified without relying on labeled data? This inquiry leads to the establishment of a new paradigm, Unsupervised Neural Architecture Search (UnNAS), which seeks to determine the feasibility of discovering high-quality neural network architectures using only image data, absent of human-annotated labels.

Research Design and Methodology

The authors employ a two-pronged approach to evaluate the effectiveness of UnNAS: sample-based experiments and search-based experiments. In the sample-based experiments, 500 randomly selected architectures from specific search spaces are trained on both supervised and unsupervised objectives. The correlation between architecture performance across these paradigms is then analyzed. In search-based experiments, a commonly used NAS algorithm, DARTS, is adapted to leverage various unsupervised objectives. The resulting architectures are assessed on tasks such as ImageNet classification and Cityscapes segmentation.

Key Findings

The study reveals several compelling insights:

• There is a consistently high rank correlation between the architectures' performance on supervised tasks and their performance on unsupervised pretext tasks. This observation holds true across various datasets, search spaces, and unsupervised tasks.
• Architectures discovered through UnNAS are competitive with those discovered using traditional, supervised NAS methods. In some cases, unsupervised search yields architectures that outperform those obtained using supervised criteria.
• The paper suggests that instead of using labeled images from smaller datasets, leveraging large datasets of unlabeled images might be more advantageous for NAS tasks.

Implications and Future Directions

These findings suggest that the salient factors determining architecture quality may be rooted in image statistics rather than in labeled data. This insight broadens the potential applications of NAS, suggesting that the burgeoning availability of unlabeled data could become a valuable resource for neural network training.

From a practical standpoint, UnNAS could simplify processes for practitioners by eliminating the need for costly label annotation, thus making neural architecture search more accessible in fields with abundant data but limited labeling resources.

The theoretical implications extend to unsupervised learning fields, hinting at a convergence where architecture and representation learning might be achieved simultaneously without direct supervision. As the findings suggest high transferability of architectures across different tasks, the pursuit of universal architectures through UnNAS is a promising avenue for future research.

While the research presents robust numerical indicators supporting UnNAS, future work might explore refining unsupervised objectives to further enhance the efficacy of NAS. Additionally, evaluating the impact of different unsupervised tasks on the versatility and adaptability of the identified architectures across a broader array of application scenarios could also provide valuable insights.

In conclusion, this research provides a substantial contribution to the neural architecture search landscape, advocating for a paradigm shift that eliminates reliance on labels and emphasizes the latent potential in image data itself.