BL19 Digital Collection Overview

• BL19 Digital Collection is a curated corpus of over 35,000 works from 1700–1899, providing granular metadata and diverse genre coverage.
• It employs a two-layer metadata architecture combining minimal PID KI records with a detailed Collection API for rapid and deep provenance lookups.
• The benchmark framework supports robust IR evaluations using models like BM25 and cross-genre RLM, demonstrating significant retrieval improvements.

The BL19 Digital Collection is a large-scale, structured corpus of over 35,000 digitized works published between 1700 and 1899 from the British Library, designed for advanced cultural analytics, information retrieval (IR), and digital collection management research. BL19 encompasses both fiction and non-fiction genres, with a focus on providing granular, provenance-aware metadata, and a robust benchmark for evaluating retrieval models and knowledge transfer from literary to factual domains (Datta et al., 17 Jan 2026, Luo et al., 2019). Its design exemplifies best practices in persistent digital object identity, collection API architectures, and evaluation methodologies for equitable, interpretable historical IR.

1. Scope, Composition, and Thematic Breadth

BL19 contains more than 35,000 digitized works, predominantly in English, and covers high-density publication periods from the 18th and 19th centuries. Two primary subsets (1830–1899) are employed for information retrieval benchmarking:

• Fiction: 10,210 titles (novels, short stories, serialized fiction), representing genres such as Gothic, sensation, social comedy, sentimental fiction, and supernatural tales.
• Non-fiction: 15,780 titles, spanning history, geography, philosophy, travel, social reform, political tracts, biographies, scientific treatises, religious and economic works.

Prominent thematic axes include social reform (e.g., Factory girls, Irish Land League), colonial and imperial topics (e.g., British adventures in India or Australia), industrialization, urban life (e.g., London fog, Chartism), moral/supernatural themes (e.g., Dracula, spiritualism), and distinctive 19th-century symbolic motifs (e.g., Italian white mice, Poor Bridget), the latter primarily in fiction (Datta et al., 17 Jan 2026).

2. Metadata Schema, Document Structure, and Provenance

BL19 employs rich, provenance-aware metadata, leveraging both British Library catalog data and Research Data Alliance (RDA) recommendations for digital object management (Luo et al., 2019). Core metadata fields are:

• Title, author, publication date (year/month granularity), publisher
• Subject headings (LoC or internal), language, shelfmark
• Document type (fiction/non-fiction), genre labels

Text is derived from OCR, stored at the page level, and then segmented into paragraphs, which form the primary retrieval and annotation unit. Preprocessing includes OCR noise correction, lowercasing, Lucene-compatible tokenization, paragraph boundary detection (blank lines/indentation), with optional stopword removal and stemming. This modular text and metadata structure enables precise, provenance-driven retrieval and supports fine-grained annotation.

The collection adopts a two-layer metadata architecture:

• PID Kernel Information (PID KI): Minimal, fixed-schema metadata (e.g., type, hadMember, wasDerivedFrom) is embedded directly in the Persistent Identifier (PID) record (Handle or DOI), enabling ultra-fast, high-level lookups without accessing the full object (Luo et al., 2019).
• Collection Metadata API: Exposes rich, arbitrarily nested collection membership and detailed provenance via a RESTful, linked-data (JSON-LD/RDF) API, suitable for deep query and traversal of complex digital object graphs.

3. Accessibility, Data Formats, and Operational API

BL19 is accessible via:

• British Library Labs (https://labs.biblios.tech) for full-text and metadata downloads (CSV, JSONL) under a research-friendly license.
• Public code and benchmark repository (https://github.com/suchanadatta/BL19-benchmark-knowledge-transfer.git).

Text data is available as plain UTF-8 (page/paragraph granularity) and in TEI-XML for rich markup needs. Metadata is provided in CSV and JSON-lines formats.

The Collection Metadata API adheres to RDA specifications, supporting core RESTful operations (create, retrieve, update, delete collections; enumerate and retrieve members), with linked-data payloads for maximal interoperability. Exemplary API calls (e.g., creating a collection with embedded PID KI, adding members, updating attributes) use shell commands and yield structured JSON-LD responses. Persistent Identifier operations (e.g., PID KI lookups) exhibit sub-millisecond latency, while Collection API operations are optimized via batched "all members" retrievals (Luo et al., 2019).

4. Provenance Distribution Strategies and Graph Performance

BL19-style collection management is informed by three core strategies for metadata and provenance distribution (Luo et al., 2019):

• I1 (PID KI-centric): All backbone provenance is in PID KI; only "hadMember" links reside in Collection API. Fastest for flat or shallow graphs (type-to-type provenance), but kernel size can increase with richer provenance.
• I2 (Hybrid): Key provenance is split; top-level in PID KI, member-level in Collection API. This minimizes PID lookups, optimizes for moderately complex ("type-to-type") graphs, and balances performance.
• I3 (API-centric): All provenance and membership stored in Collection API; PIDs remain small. Supports the richest data graphs but increases lookup latency, especially on deep graphs.

Empirical evaluation across synthetic graph shapes (G1–G4, low to high complexity) shows:

Strategy	PID Lookup (ms)	API Lookup (ms)	Optimal Use Case
I1	< 1	~8–35	Shallow/type-type graphs
I2	< 1 (PID), ~8	Balanced	Moderate hybrid topologies
I3	< 1	8–35 (dominant)	Deep/nested but tolerance for latency required

Optimal configuration for BL19: keep PID KI records minimal (root-level provenance, essential hadMember links), offload full provenance/membership to Collection API, and batch retrieval wherever possible.

5. Benchmark Construction and Evaluation Methodology

The BL19 benchmark supports reproducible, fine-grained IR evaluation in a historical/cultural context (Datta et al., 17 Jan 2026). The framework adheres to the Cranfield paradigm:

1. Document collection (fiction and non-fiction subsets)
2. 35 expert-curated queries, developed in consultation with cultural-analytics and 19th-century literature specialists, representing socially and symbolically diverse topics.
3. Paragraph-level relevance judgments: For each query, top-100 paragraphs are retrieved using BM25; experts refine candidate pools and grade relevance (0–4 scale). LLM (gpt-5-mini) grading is used for scalability, with 40% of LLM judgments manually verified (near-perfect inter-assessor agreement). Expert ratings take precedence in case of conflict.

BM25 and multiple Relevance LLM (RLM) configurations constitute the main retrieval baselines:

• base: BM25 lexical retrieval on non-fiction
• RLM_nf: pseudo-relevance feedback within non-fiction
• RLM_fn→nf: feedback trained on fiction, applied to non-fiction (cross-genre transfer)
• RLM_mix: feedback on merged corpora

Primary metrics (with LaTeX as in (Datta et al., 17 Jan 2026)):

• Precision@k: $$P@k = \frac{\#\{\text{relevant docs in top }k\}}{k}$$
• Recall@k: $$\mathrm{Recall}@k = \frac{\#\{\text{relevant docs in top }k\}}{\#\{\text{all relevant docs}\}}$$
• MAP: $$\mathrm{MAP} = \frac{1}{Q}\sum_{q=1}^Q \mathrm{AP}(q)$$
• nDCG@k: $$\mathrm{nDCG}@k = \frac{1}{Z_k}\sum_{i=1}^k \frac{2^{\mathrm{rel}_i}-1}{\log_2(i+1)}$$
• MRR: $$\mathrm{MRR} = \frac{1}{Q}\sum_{q=1}^Q \frac{1}{\mathrm{rank}_q}$$

Statistical significance is assessed by paired t-tests ($p<0.05$).

6. Key Results, Knowledge Transfer, and Analytical Insights

Benchmarks reveal that cross-genre feedback (RLM_fn→nf) yields the highest retrieval gains for scholarly/factual IR tasks, leveraging semantically rich, narrative-derived expansions from fiction:

• BM25 baseline (non-fiction): MAP=0.4993, Recall=0.5942, nDCG=0.5653, P@10=0.5122, MRR=0.6071
• RLM_nf (non-fiction feedback): MAP=0.5592 (+12%), Recall=0.6536 (+10%), nDCG=0.6059 (+7%)
• RLM_fn→nf (fiction-to-nonfiction transfer): MAP=0.5743 (+15% vs base, +3% vs RLM_nf), Recall=0.6944, nDCG=0.6183
• RLM_mix: MAP=0.5411; marginally below RLM_fn→nf, reflecting dilution of targeted signals

Improvements for RLM_fn→nf are statistically significant ($p<0.05$) and driven by the narrative, causal, and metaphoric terms latent in fiction, which fill lexical and semantic gaps in non-fiction. Per-topic analysis indicates that queries related to social reform or events (e.g., “Factory girls,” “Irish famine”) benefit most from literary context, while queries for highly symbolic motifs (“Italian white mice,” “Dublin banshee”) remain challenging across all retrieval variants.

Interpretable feedback-term analysis reveals genre-specific weighting effects: fiction emphasizes affective/narrative dimensions, while non-fiction foregrounds policy and factual discourse. Overlap terms frequently acquire divergent connotations depending on genre. The paragraph-level unit provides high annotation granularity, though deeper context or discourse-level features may further improve relevance signaling.

Limitations include the inability of lexical/probabilistic models to fully capture deep narrative structure; neural approaches with genre adaptation are indicated as a future direction. Extending evaluation to non-English works, the 18th century, and colonial literatures will test generalizability and address potential biases in access.

7. Design Principles and Methodological Implications

The BL19 digital collection demonstrably supports both high-performance, provenance-enriched digital object management and robust, interpretively sensitive IR benchmarking. Methodologically, key lessons include:

• Metadata minimization in PID KI: Only root-level, essential provenance or hadMember links are stored in PID KI for speed and immutability.
• Delegation of complex structure to Collection API: Full membership listings, deep/nested graphs, and extended provenance are exclusively handled via the Collection API.
• Rich, linked-data payloads and provenance interpretation: Consistent use of JSON-LD or RDF contexts ensures global interpretability and alignment with W3C PROV and RDA guidelines.
• Batching collection API calls: Aggregated retrieval improves performance on large or deeply nested collections.
• Hybrid feedback for IR: Cross-genre transfer from fiction offers measurable, statistically significant gains for retrieval on historical, factual queries.

A plausible implication is that adapting these principles can extend to other historical collections, enabling scalable, performant, and culturally inclusive IR infrastructures. BL19 establishes a reproducible, extensible paradigm at the intersection of digital collection curation, persistent object management, and equitable, interpretively informed information retrieval (Datta et al., 17 Jan 2026, Luo et al., 2019).