Multi-Class Subject Categorization

• Multi-Class Subject Categorization is the task of mapping items to one exclusive label from a fixed set of classes using statistical and algorithmic methods.
• Advanced techniques such as rule-based tallying, transfer learning with deep language models, and feature selection address challenges like class imbalance and semantic drift.
• Applications span bibliometrics, digital libraries, legal texts, and financial analysis, evaluated through metrics like accuracy, F1-score, and top-k accuracy.

Multi-class subject categorization is the task of assigning a single subject label from a finite set of mutually exclusive categories to each item—most commonly documents, images, or entities—using algorithmic or statistical methods. It underpins large-scale bibliometric analysis, information retrieval, digital library organization, and the design of taxonomies across scientific, legal, and commercial domains. Multi-class categorization contrasts with multi-label formulations, where each item may be assigned multiple labels, but the algorithmic approaches often share underlying representations and evaluation metrics.

1. Conceptual Foundations and Problem Formalization

Multi-class subject categorization formalizes the mapping $f: \mathcal{X} \rightarrow \mathcal{Y}$, where $\mathcal{X}$ denotes the space of item representations (e.g., text embeddings, image features) and $\mathcal{Y} = \{y_1, ..., y_K\}$ is a finite set of $K$ possible subject classes. Each item $x \in \mathcal{X}$ receives a unique label $y \in \mathcal{Y}$ according to the learned mapping. Canonical examples include assignment of Web of Science (WoS) subject categories to articles (Milojević, 2020), Mathematics Subject Classification (MSC) codes in mathematical literature (Schubotz et al., 2020), and Lipper Global categories in financial document analysis (Vamvourellis et al., 2022).

The problem is typically cast as supervised learning with a multiclass cross-entropy loss:

$$\mathcal{L}_{\mathrm{CE}}(\theta) = -\frac{1}{N} \sum_{i=1}^N \sum_{k=1}^K y_{i,k} \log \bigl( \hat{p}_{i,k} \bigr), \qquad \hat{\mathbf{p}}_i = \mathrm{softmax}(f_\theta(x_i))$$

where $y_{i,k}$ is the indicator for class $k$ (one-hot), and $f_\theta$ is the parameterized model.

Challenges include class imbalance, semantic drift between item and label granularity, the need for disambiguation where input data is under-specified, and the trade-off between prediction accuracy and interpretability.

2. Methodologies for Multi-Class Subject Categorization

A spectrum of statistical and algorithmic paradigms exists, including:

Rule-Based Reference Tallying: The Web of Science reclassification pipeline operates by counting the subject classes (SCs) of referenced articles, restricted to those published in journals with a unique, non-multidisciplinary SC. Each article is assigned the SC that appears with highest frequency in its reference list. Ties are broken by resorting to journal-level SCs or global SC frequencies. This rule-based, citation-driven assignment enforces uniqueness and is robust to the prevalence of multidisciplinary journals, providing up to 95% manual validation accuracy in external evaluation for SCs and 14 mapped broad disciplines (Milojević, 2020).

Transfer Learning with Deep LLMs: Pretrained contextualized LLMs—such as BERT and XLNet—can be fine-tuned for multi-class document categorization. Input texts are tokenized and passed through a transformer-based encoder, with the final [CLS] state projected to $K$-way output via a softmax linear layer. BERT-stage fine-tuning yields robust performance across $K=1,\dots,20$ classes, with accuracy metrics decreasing nearly linearly by ~1 percentage point per additional class, a key empirical result (Liu et al., 2019). The approach generalizes across domains including legal text (Serras et al., 2022) and financial prospectus classification (Vamvourellis et al., 2022).

Feature Selection and Classic Algorithms: For high-dimensional sparse inputs (e.g., bag-of-words text), principled feature selection is critical. The Jeffreys-Multi-Hypothesis (JMH) divergence provides an information-theoretic criterion, enabling optimal feature ranking in $O(MK)$ time complexity (for $M$ features, $K$ classes). Maximum discrimination ($MD$) and $MD-\chi^2$ methods based on JMH have been shown to improve Naive Bayes classification performance, particularly as the number of classes grows (Tang et al., 2016).

Hierarchical and Retrieval-Oriented Models: In the presence of taxonomic structures or variable label sets, models such as TagRec++ employ a dense retrieval architecture. Both inputs (e.g., questions) and hierarchical labels are embedded, and query-label similarity is computed via cross-attention mechanisms. Contrastive loss with adaptive hard negative mining yields state-of-the-art retrieval performance and supports zero-shot adaptation to new or restructured taxonomies (Viswanathan et al., 2022).

Ensemble and Non-Linear Classifiers: For applications with complex, non-linear separability—such as tweet categorization—tree-based ensemble methods (e.g., Gradient Boosting) or neural architectures (MLP, TabNet) have shown superior macro-AUC and F1 compared to linear baselines, especially on short or noisy text (Qureshi, 2021, Khademi et al., 2024).

3. Taxonomy Construction and Label System Design

Taxonomic label systems underpin subject categorization tasks. Approaches include:

• Manual and Legacy Ontologies: Schemes such as WoS SCs, MSC codes, and Lipper categories are expert-curated and periodically revised (Milojević, 2020, Schubotz et al., 2020, Vamvourellis et al., 2022).
• Collective Intelligence and Dynamic Taxonomies: Wikipedia’s category graph can be pruned and filtered via shortest path and local impact techniques to yield a large directed acyclic graph representing a multi-class, hierarchical science taxonomy. Because nodes can have multiple parents, this structure explicitly encodes multi-class inheritance and can be updated dynamically as the underlying corpus evolves (Yoon et al., 2018).
• Data-Driven Extraction and Pruning: In visual domains, subcategories may be identified via n-gram corpus mining and POS tagging, then filtered by SVMs trained on statistical and semantic features of candidate subcategories, prior to multi-instance learning (Yao et al., 2017).

The mapping from fine-grained subject classes to broader discipline groupings is often performed manually or semi-automatically, as in mapping 252 WoS SCs to 14 NSF broad areas (Milojević, 2020).

4. Evaluation Metrics and Empirical Results

Robust evaluation necessitates multiple metrics and rigorous validation procedures:

Metric	Formula	Contexts of Use
Accuracy	$\frac{TP + TN}{TP + TN + FP + FN}$	Overall correctness; limited in imbalanced data
Precision	$TP / (TP + FP)$	Emphasis on avoiding false positives
Recall	$TP / (TP + FN)$	Sensitivity to correct positive predictions
F1-score	$$2 \cdot \mathrm{Precision} \cdot \mathrm{Recall} / (\mathrm{Precision} + \mathrm{Recall})$$	Balances precision/recall; both macro/micro used
AUC-ROC	Area under ROC curve (one-vs-rest, macro/micro)	Used for multi-class to reflect ranking quality
Top-k Accuracy	Fraction where true label among the k top predicted probabilities	Important where label sets are large
Manual Validation	Direct expert judgment on sample subsets	Gold standard for stationary taxonomies

Manual validation remains essential for benchmarking in bibliometric and legal subject categorization, yielding SC-level external accuracy of 95% post-iteration in the WoS pipeline (Milojević, 2020) and micro-F1 of 0.72 in legal text (Serras et al., 2022). Cross-validation and held-out splits are standard for statistical approaches (Schubotz et al., 2020, Vamvourellis et al., 2022).

Empirical studies highlight the challenge of scaling with class cardinality: BERT-based models experienced a ~1% macro-F1 and accuracy drop per added class (K=1…20), even with balanced data (Liu et al., 2019). Feature selection methods show a relative performance increase as $K$ grows (Tang et al., 2016).

5. Domain-Specific Applications and Adaptations

Bibliometrics and Science Mapping: Reference-tallying algorithms have reclassified >40M WoS records at the article level, disambiguating within multidisciplinary journals and supporting large-scale, macro-level science studies (Milojević, 2020).

Mathematical Digital Libraries: Automatic assignment of MSC codes via multiclass logistic regression with TF–IDF representations achieves F1 close to human inter-annotator reliability, and confidence thresholding can automate ~86% of assignments at human precision levels (Schubotz et al., 2020).

Short and Noisy Text: Ensemble gradient boosting achieves superior AUC in tweet categorization across 12 general topics, especially when class boundaries are nonlinear (Qureshi, 2021). Robust preprocessing and expert annotation are critical.

Tabular and Imbalanced Medical Data: FH-TabNet leverages a multi-stage deep learning architecture with TabNet encoders to cascade from binary to refined four-way categorization, markedly improving F1 on rare disease subtypes in severely imbalanced datasets (Khademi et al., 2024).

Hierarchical and Multilingual Product Classification: Dynamic masking within transformer models (masked softmax) enforces taxonomy constraints, while bilingual input consistently improves accuracy in fine-grained product categorization tasks (Jahanshahi et al., 2021).

6. Limitations, Extensions, and Open Questions

Key limitations across methods include:

• Label System Imperfections: Legacy top-down schemes inherently inherit mis-groupings and outdated disciplinary boundaries (Milojević, 2020).
• Data Coverage: Reference-based systems require linked or citable items, leaving early or uncited work unclassified (Milojević, 2020).
• Scalability and Resource Constraints: Large transformer models (e.g., XLNet-Large) provide additional capacity at sharply increased computational costs (Liu et al., 2019).
• Multilabel and Interdisciplinarity: Most multiclass pipelines enforce single-label output; adapting to truly interdisciplinary or strongly multilabel data structures requires independent sigmoid output heads and specialized metrics (macro/micro F1) (Serras et al., 2022).
• Numeric and Hybrid Reasoning: LLMs exhibit difficulties distinguishing classes defined by numeric thresholds in texts (Vamvourellis et al., 2022).

Potential extensions are prominent across the literature:

• Incorporation of citation and reference tallies in weighted fusion (Milojević, 2020).
• Hybridizing with TF–IDF or embedding-based similarity for items lacking network links (Milojević, 2020).
• Use of dynamic, collective-intelligence-driven taxonomies to support continuous updates (Yoon et al., 2018).
• Adapting multi-label topic models to handle noisy, heterogeneous supervision sources (e.g., crowdsourcing) (Padmanabhan et al., 2016).

7. Best Practices and Practical Guidelines

Effective multi-class subject categorization systems share several characteristics:

• Use of well-defined and, if possible, dynamically updated taxonomies.
• Balanced and stratified sampling in training and validation splits to address class imbalance and label skew (Liu et al., 2019, Qureshi, 2021).
• Robust preprocessing tailored to noise properties of the input domain (e.g., stemming and lemmatization for social media, token normalization for mathematical formulae) (Qureshi, 2021, Schubotz et al., 2020).
• Model selection grounded in empirical benchmarking across a range of architectures: linear, non-linear, neural, and ensemble/statistical (Tang et al., 2016, Qureshi, 2021, Khademi et al., 2024).
• Use of confidence thresholding to control coverage-precision tradeoffs, enabling human–machine hybrid workflows for ambiguous cases (Schubotz et al., 2020).
• Complement basic accuracy with top-k accuracy, macro/micro metrics, and manual review for comprehensive evaluation (Jahanshahi et al., 2021, Serras et al., 2022).
• Consistent investment in ontology design and post-processing to mitigate the effects of label noise, sparsity, and ambiguity (Serras et al., 2022, Jahanshahi et al., 2021).

For domains subject to rapid evolution or ontology change, architectures supporting zero-shot and incremental learning of new classes via embedding-based retrieval offer long-term flexibility (Viswanathan et al., 2022).