CE-CLCNN Architecture

• CE-CLCNN is an end-to-end deep learning architecture that converts each character into a 36×36 image, enabling robust glyph-based feature extraction.
• It employs a 2D CNN for individual character embedding and a 1D CNN to capture local contextual patterns, overcoming segmentation challenges in non-segmented languages.
• Data augmentation techniques like random erasing and wildcard training enhance model robustness and mitigate overfitting in diverse character sets.

CE-CLCNN (Character Encoder–Character-Level Convolutional Neural Network) refers to an end-to-end deep learning architecture for text classification, specifically tailored for languages with large character sets and no explicit word boundaries, such as Japanese, Chinese, and Thai. The core architectural innovation is treating each character as an image, encoding it via a 2D-CNN-based “Character Encoder” and then processing sequences of these character embeddings using a separate 1D “Character-Level CNN.” The design directly addresses the overfitting and segmentation issues endemic to character-level modeling of such languages (Kitada et al., 2018).

1. Architectural Overview and Motivation

CE-CLCNN was developed to mitigate difficulties arising from traditional NLP pipelines when applied to non-segmented scripts. Word segmentation and morphological analysis are non-trivial or ill-defined in languages where word boundaries are not explicit, and standard character-based models suffer overfitting due to immense character diversity. In CE-CLCNN, each character in the input document is rendered as a fixed-resolution image (36×36 grayscale), enabling feature extraction based on visual glyph similarity as well as semantic context. The full pipeline is:

1. Convert input text (e.g., a document title) to a sequence of 36×36 grayscale images, one per character.
2. Encode each character image independently using a small 2D CNN into a 128-dimensional embedding.
3. Group the sequence into non-overlapping chunks of 10 characters (padding as needed).
4. Process per-chunk sequences of these embeddings using a 1D CNN along the character axis.
5. Map to class logits with fully connected layers and softmax for classification.
6. Train the entire system end-to-end with cross-entropy loss.

This approach enables the network to learn both visual and contextual embeddings while avoiding handcrafted segmentation and suppressing overfitting inherent in high-cardinality character vocabularies.

2. Character Encoder: 2D CNN for Glyph Embedding

The Character Encoder (CE) maps each glyph to a continuous vector in a trainable, data-driven manner:

• Input: Batch of B chunks, each chunk C=10 characters, each character as a 36×36, 1-channel (grayscale) image.
• Output: B×C×128 tensor (128-dim embedding per character).

The CE structure is as follows:

Layer	Configuration
1	Conv2D (kernel 3×3, 32 filters, stride 1, same), ReLU
2	MaxPool2D (2×2, stride 2)
3	Conv2D (3×3, 32 filters, stride 1, same), ReLU
4	MaxPool2D (2×2, stride 2)
5	Conv2D (3×3, 32 filters, stride 1, same), ReLU
6	Flatten → Linear(800→128), ReLU
7	Linear(128→128), ReLU

Key features:

• All convolutions use ReLU non-linearity; no normalization or weight decay is specified.
• Each character is processed independently (depth-wise).
• Data augmentation via Random Erasing (probability 0.3, area ratio 0.02–0.4, aspect ratio 0.3–2.0) imparts regularization and robustness to the encoder, suppressing overfitting on rare character forms.

3. Sequence Modeling: 1D Character-Level CNN

The sequence of 128-dimensional character embeddings from the CE for each chunk (size 10) is fed into a 1D CNN, termed Character-Level CNN (CLCNN), to capture local and contextual n-gram patterns:

Layer	Configuration
1	Conv1D (kernel 3, 512 filters, stride 3), ReLU
2	Conv1D (kernel 3, 512 filters, stride 3), ReLU
3	Conv1D (kernel 3, 512 filters, stride 1), ReLU
4	Conv1D (kernel 3, 512 filters, stride 1)
5	Flatten → Linear(5120→1024)
6	Linear(1024→#classes)

Notably, no explicit pooling layers are used. Temporal down-sampling is achieved via stride=3 in the initial two convolutional layers. ReLU activations are used for the first three Conv1D layers; the fourth is linear only.

4. Regularization and Data Augmentation

CE-CLCNN employs several regularization mechanisms:

• Random Erasing: Applied to each character image prior to encoding, as described above [Zhong et al. 2017].
• Wildcard Training: On the 128-dim embeddings from CE, each dimension is randomly zeroed (dropped out) with probability γ₍w₎=0.1 during training, analogous to dropout. This promotes robustness in embedding learning [Shimada et al. 2016].
• No Mentioned Additional Dropout: No extra dropout in the classification head; no weight decay or normalization layers are specified.

5. Training Procedure and Computation Details

• Batch size: 256 (number of chunks).
• Optimizer: Adam with default coefficients β₁=0.9, β₂=0.999.
• Learning rate: Not specified; typical values such as lr=1e-3 are suggested.
• Loss function: Softmax cross entropy,

$$L = -\sum_{n=1}^B \sum_{c=1}^K y_{n,c} \log \hat y_{n,c}$$

where $B$ is batch size, $K$ number of classes.

• Chunk length: 10 characters per chunk.
• Embedding dimension: 128.

Key computational operations are formally specified:

• 2D convolution for CE,

$$h_{i,j}^{(l)} = f\left(\sum_{u=-\lfloor k/2\rfloor}^{\lfloor k/2\rfloor} \sum_{v=-\lfloor k/2\rfloor}^{\lfloor k/2\rfloor} W_{u,v}^{(l)} \cdot x_{i+u,j+v}^{(l-1)} + b^{(l)}\right)$$

• 1D convolution for CLCNN,

$$h_{t}^{(l)} = f\left(\sum_{d=1}^{D_{l-1}} \sum_{\Delta=0}^{k-1} W_{d,\Delta}^{(l)} \cdot x_{d,\,t\cdot s+\Delta}^{(l-1)} + b^{(l)}\right)$$

• Max pooling for CE as usual, and cross-entropy loss as above.

6. Empirical Performance and Interpretability

CE-CLCNN demonstrates that the model can align visually and semantically similar characters within the learned embedding space, enabling generalization across rare or infrequent glyphs. Empirical results on the Wikipedia title estimation task and other open document classification datasets show that this architecture attains state-of-the-art performance compared to standard character-level and word-level models, confirming the importance of visual embedding and chunk-based local sequence modeling (Kitada et al., 2018).

7. Significance and Limitations

The CE-CLCNN approach obviates the need for handcrafted word segmentation and can directly leverage character-set similarity at the image level, a crucial property for non-segmented languages with extensive alphabets. The use of image-based embeddings is particularly effective in contexts where visually similar characters share semantic traits. However, as described, the absence of explicit normalization or advanced regularization may make model tuning dataset-dependent; hyperparameters such as chunk length and embedding size could require adaptation. Further, while the architecture is validated primarily on document titles, generalizability to long-form documents is not explicitly addressed in the provided data (Kitada et al., 2018).