Spectroscopic Redshift Ground Truth Dataset

• Spectroscopic redshift ground truth datasets are collections of astrophysical sources with precisely measured redshifts from emission and absorption line spectroscopy.
• They are constructed through targeted observational campaigns, standardized data reductions, and rigorous quality assessments that ensure reliable uncertainty estimates.
• These datasets are essential for calibrating photometric redshifts, benchmarking cosmological analyses, and anchoring astrometric frame ties in large imaging surveys.

A spectroscopic redshift ground truth dataset is a collection of astrophysical sources for which precise spectroscopic redshifts have been established using emission and/or absorption line spectroscopy. Such datasets serve as the definitive label source (“ground truth”) for calibrating, training, and validating photometric redshift estimation algorithms, benchmarking cross-correlation redshift calibration, and establishing cosmological frame ties. Spectroscopic ground truth datasets underpin the redshift calibration for cosmological weak lensing, large-scale structure studies, galaxy evolution, and astrometric reference frame alignment. The construction, validation, and scientific usage of these datasets are central to the reliability of cosmological inferences from large imaging surveys.

1. Definition and Scope of Spectroscopic Redshift Ground Truth Datasets

A spectroscopic redshift ground truth dataset comprises a homogeneous sample of astrophysical sources—typically galaxies, quasars, or compact radio sources—for which redshifts have been measured via detection and identification of spectral features against accurately calibrated rest wavelengths. Each entry is accompanied by rigorous uncertainty estimates derived from both measurement scatter and instrumental calibration, as well as quality flags to ensure reliability.

These datasets are essential for two core applications:

• Photometric redshift (photo-z) algorithm training, where spectroscopic redshifts anchor the model outputs to true distances.
• Calibration of photometric redshift distributions, often tomographically, with requirements on systematic bias, scatter, and outlier fraction set by the precision goals of dark energy and structure growth probes (Newman et al., 2013).

The relevance of spectroscopic ground truth extends from small samples that establish celestial reference frames (Titov et al., 2013) to massive compilations spanning millions of objects for cosmology (e.g., DESI DR1 (Ruggeri et al., 17 Dec 2025)).

2. Construction Methodologies

The assembly of a spectroscopic redshift ground truth dataset involves:

• Sample Selection: Defined by the scientific objective (e.g., all compact, flat-spectrum radio sources lacking optical counterparts for ICRF frame tie (Titov et al., 2013); galaxies populating under-sampled regions in multidimensional color–magnitude space for Euclid photo-z calibration (Collaboration et al., 2021, Masters et al., 2019, Collaboration et al., 2022)).
• Observational Campaigns: Employing high-resolution, broad-wavelength spectrographs (e.g., Keck DEIMOS/R ∼ 2000–3000, LRIS, MOSFIRE; Gemini GMOS; LBT LUCI; MMT Hectospec; BTA SCORPIO (Titov et al., 2013, Collaboration et al., 2021, Ebeling et al., 2014, Momcheva et al., 2015)).
• Data Reduction Pipelines: Uniform procedures including bias subtraction, flat-fielding, cosmic-ray removal, optimal extraction, wavelength calibration using arc lamps (accuracy ∼0.3–0.5 Å rms), sky subtraction, and flux calibration (Titov et al., 2013, Ebeling et al., 2014).
• Redshift Measurement: Line-by-line calculation or template cross-correlation:

$$z_i = \frac{\lambda_{\rm obs,i} - \lambda_{\rm rest,i}}{\lambda_{\rm rest,i}}$$

The final redshift $\bar{z}$ is a mean over $N$ features, and uncertainty propagation incorporates both internal scatter and calibration residuals:

$$\Delta z = \sqrt{\frac{\sigma_z^2 + (\Delta \lambda / \langle \lambda_{\rm rest} \rangle)^2}{N}}$$

(Titov et al., 2013)

• Quality Assessment: Quality/adjudication flags (e.g., multi-line feature requirement, S/N thresholds, template fit S/N, QOP codes) largely determine dataset purity and reliability (Collaboration et al., 2021, Titov et al., 2013).

3. Key Examples and Dataset Properties

Several canonical spectroscopic ground truth datasets form the benchmark for current extragalactic astrophysics and cosmology:

Dataset	N (objects)	z Range	Key Features	Reference
C3R2 DR3	5130	0.2 < z < 2.6	SOM targeting, Q≥3 redshifts, i<24.5	(Collaboration et al., 2021, Masters et al., 2019)
GalaxiesML	286,401	0.01 < z < 4	HSC grizy imaging cross-matched to secure spec-z	(Do et al., 2024)
DESI DR1 (BGS, LRG, ELG)	>13,000,000	0.1 < z < 1.6	Densest spectro-z backbone for clustering-z calibration	(Ruggeri et al., 17 Dec 2025)
IVS ICRF Sources	120 (multi-line)	0.7 < z < 3.0	VLBI compact, radio–optical frame tie, no color bias	(Titov et al., 2013)
QUBRICS	1672	2.5 < z < 5	Gaia BP/RP spectra + SED fit + ML vetting, QOP≥2	(Cristiani et al., 2023)

Each catalog provides detailed metadata: source identifiers, astrometric positions, measured redshift and error, lines/features used, S/N values, and instrumental provenance.

Example for IVS ICRF (Titov et al., 2013):

• Sample: 120 multi-emission-line AGN/QSO and radio galaxies, R<22, compact flat-spectrum radio sources.
• Success rate: ∼92% for spectroscopically attempted targets; redshift measurement uncertainty σ_z ≤ 0.002 for single-line, even lower for multi-line sources.
• Key purpose: radio–optical–Gaia reference frame anchoring.

Example for GalaxiesML (Do et al., 2024):

• Sample: 286,401 galaxies, HSC grizy photometry + cross-matched public spec-z.
• Redshift distribution: peaks at z ≈ 1.5, drops rapidly beyond z ≈ 2.5.
• Selection: S/N cuts, secure spec-z flag, magnitude/quality filtering, duplicate removal.

4. Validation, Metrics, and Precision

The accuracy and representativity of spectroscopic ground truth datasets are characterized by:

• Measurement Uncertainties: Typically σ_z < 0.001 per object for robust multi-feature redshifts, with σ_z ≈ 0.0002 typical for state-of-the-art surveys (Soriano et al., 2024, Do et al., 2024). Empirical repeat observations are used to verify reported uncertainties (Momcheva et al., 2015, Ebeling et al., 2014).
• Systematics and Selection Biases: Incompleteness (e.g., color, magnitude, or morphological selection) and inhomogeneous sky coverage are major sources of bias, which can propagate into training and calibration (Newman et al., 2013). Best practices include targeting underrepresented color–magnitude cells via SOM mapping (Collaboration et al., 2021), explicit reporting of failures, and quantification of sample completeness.
• Validation Metrics: Metrics for benchmarking photometric and machine-learning redshifts against the spectroscopic ground truth typically include:
  • Normalized bias:

  $$\mathrm{Bias} = \left\langle \frac{z_{\rm phot} - z_{\rm spec}}{1 + z_{\rm spec}} \right\rangle$$ - Scatter (normalized median absolute deviation):

  $$\sigma_{\rm MAD} = 1.48\,\mathrm{Median} \left[ \left| \frac{z_{\rm phot} - z_{\rm spec}}{1 + z_{\rm spec}} - \mathrm{Median}(\cdot) \right| \right]$$ - Outlier fraction:

  $$f_{\rm out} = \frac{\# \{ |\delta z| > 0.15 \} }{N_{\rm total}}$$

  with $$\delta z = (z_{\rm phot} - z_{\rm spec})/(1 + z_{\rm spec})$$ (Do et al., 2024, Collaboration et al., 2021).

• Cosmological Requirements: For Stage IV dark energy surveys, systematic bias of mean redshift per tomographic bin must satisfy $$|\langle z_\mathrm{phot} - z_\mathrm{spec} \rangle| \leq 2\times 10^{-3} (1+z)$$ and $f_{\rm out} \lesssim 0.1\%$ per bin (Newman et al., 2013).

5. Scientific Applications

Spectroscopic ground truth catalogs have several primary applications:

• Photo-z Training and Calibration: Core to the empirical mapping from photometry (colors, magnitudes, structural parameters) to redshift. Techniques include regression training sets, calibration of $P(z|C)$, validation of model uncertainty, and construction of empirical redshift priors (Do et al., 2024, Collaboration et al., 2021, Ebeling et al., 2014).
• Cross-correlation (clustering-z) Calibration: Enables reconstruction of photometric sample redshift distributions via cross-correlation with spectroscopic “reference” samples, crucial when direct spectroscopic completeness is unattainable (Ruggeri et al., 17 Dec 2025). The clustering-z formalism requires dense, well-understood spec-z reference datasets spanning the relevant cosmic volume.
• Astrometric Frame Ties and Proper Motion Studies: Linkage of radio (VLBI) and optical (Gaia) frames via accurate redshifts for compact sources (Titov et al., 2013).
• Cosmic Chronometers and Redshift Drift: Datasets such as QUBRICS provide bright, high-z QSO anchors for Sandage-test experiments (Cristiani et al., 2023).
• Instrumental and Photometric Systematics Diagnostics: Matched photometric-spectroscopic catalogs facilitate aperture-matching, PSF correction, and the identification/mitigation of color-dependent biases (Zhou et al., 2019).

6. Limitations, Biases, and Best Practices

Spectroscopic ground truth datasets are constrained by:

• Selection Effects: Magnitude-limited, color- or morphology-selected samples miss certain galaxy types or redshift/magnitude regimes. Under-representation of faint, red, or high-z sources can bias calibration (Masters et al., 2019, Collaboration et al., 2021).
• Spectroscopic Incompleteness: Redshift failures and ambiguous features disproportionately affect the faint end, blue galaxies, or certain redshift ranges (e.g., the “redshift desert” at 1.4 < z < 1.8) (Collaboration et al., 2021, Collaboration et al., 2022).
• Field-to-field Cosmic Variance: Mitigated by observing many widely separated fields of sufficient area to sample galaxy population variations (Newman et al., 2013).
• Instrumental Effects: Flux calibration, wavelength solution errors, and spatial sampling variations must be controlled and propagated to final catalog uncertainties.

Best practices include:

• Explicit quality flagging (multi-feature or strong-line redshift requirement).
• Documented sample selection and completeness.
• Reproducible, machine-readable catalog schemas including measurement and observational metadata.
• Release of both spectroscopic and photometric metadata for cross-validation.
• Use of self-organizing maps (SOM) or similar high-dimensional approaches to optimize spectroscopic targeting (Collaboration et al., 2021, Collaboration et al., 2022).

7. Role in Machine Learning and Future Surveys

Large spectroscopic ground truth datasets now play a central role in machine learning for astrophysical inference:

• Serve as the “label space” for deep learning and ensemble approaches combining imaging, photometry, and structural features to regress redshift with quantifiable uncertainty (Do et al., 2024, Seenivasan et al., 1 Jan 2026, Soriano et al., 2024).
• Enable advanced transfer learning, domain adaptation, and uncertainty calibration, as demonstrated by LoRA fine-tuning and joint training frameworks (Seenivasan et al., 1 Jan 2026, Soriano et al., 2024).
• Inform optimal observing strategies for upcoming surveys such as Euclid, Roman, LSST, and SPHEREx, where photometric datasets will vastly exceed available spectroscopic coverage, necessitating efficient ground truth utilization for robust photo-z estimation and cosmological calibration (Newman et al., 2013, Collaboration et al., 2021, Ruggeri et al., 17 Dec 2025).

Spectroscopic redshift ground truth datasets will continue to be the standard against which all photometric and algorithmic redshift estimators are benchmarked, underpinning the precision cosmology program for the coming decades.