LitVISTA: Narrative Orchestration Benchmark

• LitVISTA is a benchmark and framework that defines narrative orchestration through a formal narrative space (VISTA Space) and a richly annotated corpus.
• It operationalizes narrative anchors into distinct roles—impulses, resonances, and pauses—using a dependency graph to encode plot and pacing.
• Experimental evaluations reveal that even advanced LLMs struggle with integrating event functions and long-range dependencies, highlighting a gap in narrative comprehension.

LitVISTA is a benchmark and theoretical framework designed to enable systematic computational evaluation of narrative orchestration in literary texts. Addressing the structural gap between models' coherence-centric story generation and the nuanced orchestration of plot, pacing, and intensity inherent in human narrative, LitVISTA operationalizes a formal narrative space—VISTA Space—and provides an extensively annotated corpus for model assessment. Experiments using oracle-level annotations on frontier LLMs reveal core limitations in current models' capacity to jointly capture narrative function and structure, establishing LitVISTA as a critical resource for advancing narrative understanding in NLP (Lu et al., 10 Jan 2026).

1. Formalism: VISTA Space

VISTA Space constitutes a high-dimensional representational framework that encodes narrative orchestration with mathematical rigor. The core formalism comprises:

• Narrative Anchors: Minimal tokens, termed "Verb⁺", consisting of finite verbs or event-denoting nominals (e.g., marriage, departure), serve as proxies for narrative events and bridge surface text to abstract structure.
• Anchors' Dynamic Roles: Each Verb⁺ is assigned one of three topological roles:
  • Impulses ($V_I$): Plot-advancing events
  • Resonances ($V_R$): Descriptive expansions modulating rhythm and tension
  • Pauses ($V_P$): Intensity-building suspensions
  • Non-events ($V_\emptyset$): Syntactic or filler tokens are explicitly excluded from orchestration
• Narrative Coordinates:
  • Narrative Progress Index $\tau \in \mathbb{N}$: Discrete stage counter
  • Marginal Increment $\delta \in (0,1) \subset \mathbb{R}$: Continuous micro-shift
  • Metric Domain: $\tau \in \mathbb{N},\ \delta \in (0,1)$
• Anchor Topology: For each anchor $v$, a state transition $\mathcal{F}(v)$ is defined:

  $$\mathcal{F}(v) \in \{E_\tau \rightarrow E_{\tau+1},\ E_\tau \rightarrow E_{\tau+\delta},\ E_\tau \rightarrow E_\tau\}$$

  with transitions corresponding to impulses, resonances, and pauses, respectively.

• Narrative Dependency Graph: G = (V, E), with

  $$E \subseteq (V_R\times V_I) \cup (V_P\times (V_I \cup V_R))$$

  encoding directed dependencies where resonances and pauses link to storyline-advancing impulses or expansions.

• VISTA 3D Space: The orchestration space is represented along axes of Impulses ($\tau$), Resonances ($N\delta$), and Pauses (unit amplitude), enabling projection of both human and model narrative traces for comparison.

2. LitVISTA Benchmark Construction

LitVISTA is constructed to provide a structurally annotated dataset reflecting authentic literary narrative complexity:

• Source Corpus: Derived from LitBank, comprising over 100 public-domain English literary works, utilizing entire narrative chapters to preserve long-range structures.
• Annotation Schema:
  • Anchor Layer: Inherits "Verb⁺" triggers from LitBank.
  • Topological Role Labels: Each anchor receives one of three roles (V_I, V_R, V_P); non-events are excluded.
  • Directed Dependencies: As per the VISTA-space dependency graph.
• Multi-Phase Annotation (Figure 1 in (Lu et al., 10 Jan 2026)):
  • Phase 1: Independent anchor role annotation by two experts ($\kappa = 0.49$).
  • Phase 2: Edge annotation by a second pair ($\kappa = 0.76$).
  • Phase 3: Senior adjudicators resolve disagreements to yield the gold-standard graph, informed by a detailed manual and theoretical codebook.
• Corpus Statistics:
  • Average chapter length: ~10,000 tokens
  • Average anchor counts per chapter: $|V_I| \approx 13$–$18$; $|V_R| \approx 50$–$80$; $|V_P| \approx 3$–$4$
  • Average cross-dependencies: 60–100 per chapter
  • Data split: 8:1:1 (train:validation:test)

Statistic	Value per Chapter	Notes
Tokens	$\sim$10,000	Long-range narrative preserved
Impulses ($V_I$)	13–18	Plot advances
Resonances ($V_R$)	50–80	Descriptive/microevents
Pauses ($V_P$)	3–4	Suspension/intensity
Cross-dependencies	60–100	Structural complexity

3. Evaluation Protocol and Metrics

The evaluation decomposes narrative parsing into event-level role assignment and dependency recovery:

• Task Definition: For a given text $T$ and gold anchor set $V_{\text{cand}}$, models must:
  1. Predict each anchor's role ($r\in\{V_I, V_R, V_P\}$).
  2. For each anchor, identify its "head" ($u$) among $V_{\text{cand}} \setminus \{v\}$ to reconstruct the dependency.

• Joint Optimization Objective:

  $$\begin{cases} r^* = \arg\max_r P(r\,|\,v, T) \ u^* = \arg\max_{u\neq v} P(v \rightarrow u\,|\,v, r^*, T) \end{cases}$$

• Metrics:
  • Role Classification: Precision, Recall, F1 for anchor roles
  • Dependency Parsing: Precision, Recall, F1 for directed edges
  • Overall Score: Harmonic mean of the six F1 scores (role and dependency)

4. Experimental Evaluation and Analysis

Frontier LLMs (GPT, Claude, Grok, Gemini) are evaluated via oracle tasks with the following findings:

• Performance Landscape:
  • Marked asymmetry: models often excel in only one task—either anchor role classification or dependency parsing.
  • Very few models achieve strong results in both; top performance obtained by Claude-sonnet-4-thinking (Anchor F1 ≈ 0.49, Dep F1 ≈ 0.56, Harmonic ≈ 0.52), followed by GPT-5.2 (Anchor F1 ≈ 0.40, Dep F1 ≈ 0.36, Harmonic ≈ 0.38).
• Effects of "Thinking" Modes:
  • Enabling chain-of-thought or advanced reasoning redistributes performance rather than yielding uniform improvement.
  • For instance, GPT-5.1-thinking achieves Dep F1 = 0.68 but Anchor F1 drops to 0.24. Some families (e.g., Claude) display coherent shifts; others (e.g., GPT, Grok) show disparate trajectory changes.
  • Conclusion: "thinking" amplifies models' inherent biases rather than rectifying systemic narrative deficiencies.
• Qualitative Deficiencies:
  • Universal inability to form a unified global narrative view; models cannot coordinate event function (roles) with higher-order structure (dependencies).
  • Failures are especially pronounced with long-range dependencies, spanning hundreds to thousands of tokens.
  • Lexical distinctions cluster by role, but models are unable to exploit these lexicons to construct coherent dependency topologies.

5. Implications and Prospective Research Directions

The operationalization of narrative orchestration in VISTA Space and the LitVISTA benchmark motivates several research trajectories:

• Bridging Long-Range Dependencies: Develop neural architectures or objectives tailored to capture non-local narrative links (e.g., graph transformers over the Verb⁺ layer).
• Unified Representation Learning: Pretrain models directly on VISTA-style topologies to foster alignment between human and model narrative structures.
• Structured Prompting and Fine-Tuning: Incorporate VISTA axioms and codebook-guided schemas into prompt design and model adaptation.
• Multidimensional Narrativity: Extend the role taxonomy beyond impulses, resonances, and pauses, integrating finer-grained rhetorical phenomena such as foreshadowing or flashback.
• Cross-Domain and Multilingual Expansion: Adapt LitVISTA annotation and evaluation to a broader array of narrative forms (e.g., modern fiction, stage/screen scripts) and languages.
• End-to-End Robustness: Design narrative generation paradigms that guarantee comprehensive anchor extraction and precise structural alignment, leveraging retrieval-augmented and constrained decoding.

This suggests a growing recognition that narrative comprehension in LLMs demands inductive biases and architectures capable of encoding structural, not just local or surface, relationships.

6. Significance and Theoretical Contribution

LitVISTA establishes a rigorous framework for characterizing and evaluating narrative orchestration, providing both the mathematical scaffold (VISTA Space) and the empirical resource (benchmark corpus) to highlight systemic deficiencies in current LLMs. The finding that these models cannot recover unified global arcs or manage long-range functional-structural coordination at an oracle level highlights a substantial gap in computational narratology and story generation. The benchmark thereby catalyzes further inquiry into architectures and learning protocols that "think like readers" with respect to plot, rhythm, and literary pacing (Lu et al., 10 Jan 2026).