Unsupervised Learning of Artistic Styles with Archetypal Style Analysis

Abstract: In this paper, we introduce an unsupervised learning approach to automatically discover, summarize, and manipulate artistic styles from large collections of paintings. Our method is based on archetypal analysis, which is an unsupervised learning technique akin to sparse coding with a geometric interpretation. When applied to deep image representations from a collection of artworks, it learns a dictionary of archetypal styles, which can be easily visualized. After training the model, the style of a new image, which is characterized by local statistics of deep visual features, is approximated by a sparse convex combination of archetypes. This enables us to interpret which archetypal styles are present in the input image, and in which proportion. Finally, our approach allows us to manipulate the coefficients of the latent archetypal decomposition, and achieve various special effects such as style enhancement, transfer, and interpolation between multiple archetypes.

• The paper introduces an unsupervised framework that identifies and decomposes artistic styles using archetypal analysis on deep image features.
• The method leverages sparse convex combinations to represent styles as interpretable archetypes, enabling nuanced style transformations.
• Experimental results on diverse datasets demonstrate improved content preservation and enriched style effects compared to traditional methods.

Unsupervised Learning of Artistic Styles with Archetypal Style Analysis

Introduction

The paper explores unsupervised techniques for discovering and manipulating artistic styles in large collections of paintings through archetypal style analysis. Unlike prior efforts aimed at improving style transfer methods, this research focuses on the automatic identification and summarization of style characteristics using archetypal analysis, an unsupervised learning technique developed with interpretability in mind. By leveraging deep image representations, the technique establishes a repertoire of archetypal styles that provide a robust framework for artistic style manipulation.

Archetypal Style Analysis

The cornerstone of the approach is archetypal style analysis, which enables the unsupervised learning of style representations. The method uses a deep convolutional neural network to extract high-dimensional style features from paintings, subsequently reduced via singular value decomposition to capture the main variance. The stylistic essence of paintings is encoded as sparse convex combinations of archetypes, each representing a visual archetype that can be decomposed into meaningful aesthetic components.

Figure 1: Using deep archetypal style analysis, we can represent an artistic image (a) as a convex combination of archetypes. The archetypes can be visualized as synthesized textures (b), as a convex combination of artworks (c) or, when analyzing a specific image, as stylized versions of that image itself (d). Free recombination of the archetypal styles then allows for novel stylizations of the input.

Archetypal Style Manipulation

Building upon the universal style transfer methodology, the paper introducers modifications to enhance content details preservation within stylized images. Manipulating archetypal decompositions allows for complex style transformations, such as style enhancement and interpolation, achieved by adjusting the sparse coding coefficients linked to archetypes. This technique empowers artists and researchers to creatively modify styles, producing novel visual effects through affine transformations of archetype-driven style matrices.

Experiments

The paper presents several experimental results on two datasets: a large collection named GanGogh and paintings of Vincent van Gogh. Archetypes are synthesized using style representations to visualize textures and display the strongest painterly contributions, showcasing the varieties of styles across different artistic periods.

Figure 2: Picasso's "Pitcher and Fruit Bowl".

Figure 3: "Les Alpilles, Mountain Landscape near South-Reme" by van Gogh, from the van Gogh collection.

The experiments critically compare the refined stylization approach against previously established methods, proving substantial improvements in content preservation while enhancing style effects. Furthermore, visualization aids in deciphering archetypal presence in each painting, effectively conveying style diversity through meaningful interpolation between archetypes.

Discussion

Archetypal style analysis emerges as a versatile tool for unsupervised style discovery and manipulation with applications extending beyond artistic transformation to understanding stylistic evolution in visual art. The dual interpretation of archetypes as composite and interpretative elements underscores its value in applied machine learning settings, facilitating deeper stylistic insights. Future work aims to bridge these styles with art historical metadata to enrich understanding and contextual relevance in interdisciplinary contexts.

Conclusion

The research introduces a potent unsupervised framework for artistic style manipulation, underscored by archetypal style analysis' interpretability. The technique paves the way for strategic explorations in artistic styles, enabling both visuals and meta-analyses of artistic genres. As a future trajectory, integrating metadata with archetypal insights offers transformative implications in digital humanities, propelling deeper artistic investigations.