Controllable Artistic Text Style Transfer via Shape-Matching GAN

Abstract: Artistic text style transfer is the task of migrating the style from a source image to the target text to create artistic typography. Recent style transfer methods have considered texture control to enhance usability. However, controlling the stylistic degree in terms of shape deformation remains an important open challenge. In this paper, we present the first text style transfer network that allows for real-time control of the crucial stylistic degree of the glyph through an adjustable parameter. Our key contribution is a novel bidirectional shape matching framework to establish an effective glyph-style mapping at various deformation levels without paired ground truth. Based on this idea, we propose a scale-controllable module to empower a single network to continuously characterize the multi-scale shape features of the style image and transfer these features to the target text. The proposed method demonstrates its superiority over previous state-of-the-arts in generating diverse, controllable and high-quality stylized text.

• The paper introduces a bidirectional shape matching framework enabling continuous control over glyph deformations to balance legibility and artistry.
• It features a sketch module that transforms a single style image into a multi-scale dataset, capturing diverse deformation levels effectively.
• The Controllable ResBlock in the GAN achieves smooth transitions across stylistic deformations, outperforming state-of-the-art models in both visual appeal and clarity.

Controllable Artistic Text Style Transfer via Shape-Matching GAN

This paper addresses the challenge of artistic text style transfer, focusing on real-time control of glyph deformation to produce stylized typography from a single source image. The authors introduce Shape-Matching GAN, a framework that facilitates continuous and controllable adjustment of stylistic deformations in text.

Key Contributions

1. Bidirectional Shape Matching Framework: The paper proposes a novel framework allowing for scalable control over glyph deformation, which is crucial for maintaining the balance between legibility and artistry. This framework establishes a robust mapping between style images and target glyphs by employing a bidirectional shape matching strategy.
2. Sketch Module: A unique sketch module is introduced to effectively transform a single style image into a multi-scale training dataset. This module captures shape features across different deformation levels, thus enabling robust glyph-style mapping.
3. Scale-Controllable Network: Utilizing a Controllable ResBlock, the proposed GAN model adapts to various levels of stylistic deformations, providing smooth transitions across styles without needing model retraining.

Methodology

The method divides the task into two main processes: structure transfer and texture transfer. The structure transfer involves a backward-forward method for shape matching, where contour simplification in the backward phase supports the transfer of style features to target glyphs. In the forward phase, the Shape-Matching GAN applies artistic style through learned coarse-to-fine mappings.

The texture transfer utilizes an adversarial training setup to synthesize the final artistic text, which ensures both structural fidelity and stylistic richness.

Empirical Evaluation

The proposed method shows clear superiority when compared to state-of-the-art style transfer models. It demonstrates better performance in terms of both artistic expression and preservation of text legibility, as illustrated by user studies and qualitative comparisons. The ability of the model to adjust glyph deformation continuously offers substantial advantages over existing discrete-level methods.

Implications and Future Work

Practically, this work has significant implications for dynamic typography and graphic design by offering real-time interactiveness in artistic text rendering. Theoretically, it provides a robust framework for future work in multi-scale and controllable style transfer.

The model holds promise for further exploration in adapting the smoothness block within the sketch module for more style versatility and could be extended to dynamic applications such as text video synthesis. Moreover, the disentanglement of structure and texture offers intriguing possibilities for broader applications beyond text, such as symbol stylization and icon design.

Overall, this paper presents a substantial contribution to the field of artistic style transfer, providing a practical avenue for further research and development in AI-driven visual creativity.