Harmful Essay Detection Benchmark

• The paper presents a rigorous framework for evaluating harmful content detection in long essays by formalizing evolving harm taxonomies.
• It details dataset construction protocols, balanced annotations from varied sources, and multi-task Transformer-based architectures for both binary and multi-label classification.
• Evaluation metrics focus on fairness, bias detection, and operational robustness, offering actionable insights for ethical natural language processing.

Harmful Essay Detection (HED) Benchmark is a rigorous framework for evaluating automated systems' ability to identify and characterize harmful content in long-form text, particularly essays. HED benchmarks draw explicitly from evolving taxonomies of harm in natural language processing, targeting both overt and subtle manifestations of toxicity, bias, untruthfulness, and ethical violations. The benchmark supports application domains ranging from education (student essays) to social moderation and automated essay scoring, with an emphasis on transparency, fairness across demographic groups, and operational robustness (Rauh et al., 2022, Valencia et al., 2021, Kim et al., 9 Jan 2026, Liu et al., 12 Jun 2025).

1. Harm Taxonomy and Formalization

HED benchmarks encode harm evaluation via six foundational characteristics (Rauh et al., 2022):

1. Harm Definition: Harm is operationalized as the real-world effect a given essay may exert, represented by $H(e)$ (binary indicator for essay $e$). Detection systems aim to approximate $H(e)$ by scoring each essay $\hat{y}(e)$, which can be binary, continuous, or multi-label.
2. Representation, Allocation, Capability:
   • Representational harm ($R$): Negative or unfair depictions based on identity. Measured by $\Delta R(g,g') = E[Y|g] - E[Y|g']$ for groups $g,g'$.
   • Allocational harm ($A$): Unequal distribution of resources, formalized as $\Delta A(g,g') = P(resource|g) - P(resource|g')$.
   • Capability fairness ($C$): Model performance gap $\Delta C(g,g') = Perf(g) - Perf(g')$.
3. Instance vs Distributional Harm:
   • Instance: Harm emerges from a single essay exceeding a threshold score.
   • Distributional: Aggregate harm across a corpus or population, examined by the distribution $\{ \hat{y}(e) \}$.
4. Context (Textual, Application, Social):
   • Evaluations must specify textual length ($L$), conditioning ($P$), and social norm setting ($S$).
   • Contextual sensitivity includes full-document scoring, scenario-dependent annotation instructions, and annotator demographic documentation.
5. Harm Recipient:
   • Annotation must clarify whether harm is likely to affect the subject mentioned, the reader, the author persona, or society ($H_r(e)$ for $r \in \{S, R, A, SO\}$).
6. Demographic Groups:
   • Harm evaluation should condition on protected attributes ($A \in \{a_1, ..., a_k\}$), enabling fairness and bias analysis.

A plausible implication is that multi-faceted annotation schemas are mandatory for benchmark reliability.

2. Dataset Construction Protocols

Benchmark datasets comprise both real and synthetic essays from varied domains (education, social media, model-generated text), with careful class balancing (Rauh et al., 2022, Kim et al., 9 Jan 2026, Valencia et al., 2021, Liu et al., 12 Jun 2025).

Sourcing:

• Educational essays (ACT, IELTS, birth cohort studies) for standardized detection (Valencia et al., 2021, Kim et al., 9 Jan 2026).
• Social-media long-form posts and commentaries (e.g., WeChat, blogs) for broader domain coverage (Liu et al., 12 Jun 2025).
• Synthetic data via LLMs and teacher–student frameworks to simulate rare or adversarial harm scenarios (Kim et al., 9 Jan 2026, Liu et al., 12 Jun 2025).

Class Balance:

• Typical schema is 50/50 harmful vs non-harmful essays. Harmful categories may include identity insults, violence incitement, stereotypes, misinformation, and policy violations (e.g., gambling, pornography, abuse, fraud) (Rauh et al., 2022, Liu et al., 12 Jun 2025).

Annotation:

• Binary labels (harmful/non-harmful) supplemented with multi-class harm types, harm spans, harm recipients, and severity ratings.
• Annotators are given detailed guidelines, examples, qualification tests, and must maintain $\geq 80\%$ agreement. Inter-annotator agreement (Cohen’s $\kappa$, Fleiss’ $\kappa$) is formally monitored (Rauh et al., 2022, Liu et al., 12 Jun 2025).
• Final labels are determined by majority vote; expert adjudication resolves ambiguous cases.

3. Modeling Architectures and Training

HED system architectures primarily leverage pre-trained Transformers, with multi-task heads and knowledge-base augmentation (Valencia et al., 2021, Liu et al., 12 Jun 2025).

Approaches:

• BERT-Base, RoBERTa, ELECTRA, and Llama variants, deployed for both binary and multi-label harm classification (Valencia et al., 2021, Kim et al., 9 Jan 2026).
• Head architectures: separate heads per harm category, binary head for overall detection. Activation functions include ReLU.
• Multi-task recipes: Individual heads are fine-tuned on different auxiliary sources (toxic comment, emotion, essay regression), combined via logistic regression or multi-task loss:

$$\mathcal{L} = \sum_{c=1}^{5} \lambda_c \mathcal{L}_{\textrm{CE}}^{(c)} + \lambda_{\textrm{bin}} \mathcal{L}_{\textrm{CE}}^{(bin)}$$

Knowledge-Augmentation:

• Explicit rule bases (keyword lists, regular expressions, domain heuristics) are incorporated in prompts and supervised fine-tuning to improve detection of subtle or evasive harms; these augment both zero-shot and fine-tuning regimes, yielding substantial macro-F1 gains (+0.15–0.25) in Chinese benchmarks (Liu et al., 12 Jun 2025).

Handling Long-form Essays:

• Hierarchical encoders segment long essays into token windows (e.g. 50–512 tokens), with sentence-level or paragraph-level analysis; window representations are processed via attention-based classifiers or majority vote (Liu et al., 12 Jun 2025).

4. Evaluation Metrics and Reporting

HED benchmarks mandate multi-level, fairness-sensitive reporting standards (Rauh et al., 2022, Valencia et al., 2021, Kim et al., 9 Jan 2026, Liu et al., 12 Jun 2025).

Core Metrics:

• Precision, recall, F1 per class:

$$\textrm{Precision}_c = \frac{\textrm{TP}_c}{\textrm{TP}_c + \textrm{FP}_c}, \quad \textrm{Recall}_c = \frac{\textrm{TP}_c}{\textrm{TP}_c + \textrm{FN}_c}, \quad \textrm{F1}_c = 2 \cdot \frac{\textrm{Precision}_c \cdot \textrm{Recall}_c}{\textrm{Precision}_c + \textrm{Recall}_c}$$

• Macro-F1: equal-weight average for multiclass harm.
• ROC-AUC: area under the ROC curve, often reported for binary detection.
• Quadratic Weighted Kappa (QWK): agreement metric for essay scoring (Kim et al., 9 Jan 2026).
• Distributional: fraction of harmful sentences $F(e)$; instance vs aggregate reporting.

Fairness and Bias:

• Demographic parity gap $$\Delta_{\textrm{DP}} = |P(\hat{y}=1|A=a) - P(\hat{y}=1|A=b)|$$.
• Equality of opportunity gap $\Delta_{\textrm{EO}} = |TPR(a) - TPR(b)|$.
• Representational bias score (harm spans ratio across groups).

Reporting Standards:

• Per-group breakdowns for all metrics, confusion matrices by group and harm type, bootstrap confidence intervals, and qualitative error exemplars (Rauh et al., 2022).

5. Comparative Performance and Error Analysis

Empirical analysis across HED studies provides insight into model robustness, bias, and residual weaknesses.

Automated Systems:

• Instruction-tuned LLMs (Llama3 series) resist generating harmful content (POR=100%) and outperform standard LLMs in harmful-vs-argumentative essay discrimination (Macro F1≈79) (Kim et al., 9 Jan 2026).
• Persona injection (altering race, character) induces or mitigates bias, with classification performance fluctuating by up to 7 F1 points across demographic personas (Kim et al., 9 Jan 2026).
• AES models and LLMs systematically over-score harmful essays relative to benign argumentative ones, reflecting lack of ethical context integration (Kim et al., 9 Jan 2026).
• Feature analysis pinpoints suicide, severe toxicity, sadness as strong positive signals; optimism, trust, control as negatives (Valencia et al., 2021).

Challenges:

• Lower recall for Harmful class across all models: hatefully-structured argumentative essays are frequently misclassified (Kim et al., 9 Jan 2026).
• Inter-annotator agreement remains an open challenge in all benchmarks (Valencia et al., 2021).
• Small datasets and absence of fine-grained categories hinder generalization and sensitivity (Valencia et al., 2021).
• Synthetic data generation and rule bases improve lightweight classifier performance, matching SOTA LLMs with fewer resources (Liu et al., 12 Jun 2025).

6. Practical and Ethical Implications

HED benchmarks expose substantive gaps in both model and annotation practice. Lack of explicit ethical context in scoring functions can inadvertently validate harmful worldviews, especially in automated essay assessment scenarios (Kim et al., 9 Jan 2026). Benchmarks that explicitly incorporate harm annotation guidelines into model scoring instructions markedly correct this tendency, lowering harmful-essay scores and improving reliability (Kim et al., 9 Jan 2026). Ongoing calibration of alert thresholds, continual rule-base updates, and context-preserving architectures are essential for operational fairness and error mitigation (Rauh et al., 2022, Liu et al., 12 Jun 2025).

7. Extending HED to Multilingual and Domain-specific Scenarios

Recent benchmarks demonstrate the adaptability of HED principles beyond English-language corpora. ChineseHarm-Bench introduces category-specific rule bases, knowledge-augmented prompt strategies, and hierarchical encoding for essay-length moderation in Chinese domains, successfully aligning lightweight detectors with state-of-the-art LLMs through explicit pattern transfer (Liu et al., 12 Jun 2025). This suggests that future HED benchmarks will generalize across linguistic and topical boundaries by hybridizing real-data annotation, synthetic data simulation, and continual knowledge-injection.

---

HED Benchmarks formalize the technical, operational, and ethical apparatus necessary for robust, fair, and transparent detection of harmful long-form text. Their continual evolution reflects interdisciplinary methodology, spanning computational linguistics, machine learning, social policy, and educational measurement (Rauh et al., 2022, Valencia et al., 2021, Kim et al., 9 Jan 2026, Liu et al., 12 Jun 2025).