SKA Regional Centre Network (SRCNet)

• SRCNet is an international federation of data centres that store, process, and analyze exabyte-scale data from the SKA Observatory.
• It integrates in-situ computing with FaaS, Kubernetes-managed clusters, and semantic interoperability to optimize data access and reduce transfers.
• The network employs advanced replication, provenance capture, and automated governance to ensure scalable, reproducible astronomical research.

The SKA Regional Centre Network (SRCNet) is an international federation of geographically distributed data centres tasked with the storage, curation, processing, and analysis of exabyte-scale data products generated by the Square Kilometre Array (SKA) Observatory. Designed to overcome the prohibitive network, storage, and computational bottlenecks associated with centralised data handling, SRCNet incorporates a mesh of high-performance facilities, federated services, and advanced in-situ computing paradigms—including Function-as-a-Service (FaaS), workflow orchestration, semantic interoperability models, and rigorous provenance capture—to make global, reproducible astronomical science feasible at the scale of hundreds of petabytes to exabytes per year (Parra-Royón et al., 5 Jan 2026, João et al., 19 Dec 2025, Possenti, 4 Apr 2025).

1. Federation Architecture and Data Flow

SRCNet is architected as a dynamically federated mesh of SRC nodes, each acting both as a data repository and a compute centre. Data originates from SKA telescopes in Australia and South Africa, where pre-processing at Science Processing Centres (SPCs) produces Level 1 science products. These products are transferred via high-throughput optical networks into a global Datalake (implemented with Rucio), which replicates data to multiple SRC nodes for redundancy and access locality (Parra-Royón et al., 12 Jan 2026, Parra-Royón et al., 2023).

Each SRCNet node hosts a storage backend (Rucio Storage Element, RSE), a Kubernetes-managed compute cluster (supporting CPU, GPU, ARM, and accelerators), and a suite of user- and workflow-facing services. Federation is managed through a bifurcated service model:

• Global services—identity and access management (SKAO IAM), permissions and policy APIs, site capability catalogues, and global data- and workflow-discovery (via Rucio, IVOA DataLink, and semantic models).
• Local services—data access and execution endpoints, GateKeeper-protected ingress, and Kubernetes FaaS runtimes.

Data volumes routinely exceed 700 PB/year, with aggregate network egress peaking at several PB/day during distribution bursts. Rucio enforces ≥2 replica policies per data product, optimising replica placement by science demand and network topology (Parra-Royón et al., 12 Jan 2026, Parra-Royón et al., 2023).

2. In-situ Computing and Function-as-a-Service Integration

To address the infeasibility of moving petascale-to-exascale data across international networks for central analysis, SRCNet adopts an explicitly computation-to-data model. This paradigm leverages in-situ function execution proximate to data replicas, minimising inter-node transfers, reducing latency, and exploiting data-local accelerators (Parra-Royón et al., 5 Jan 2026, Parra-Royón et al., 12 Jan 2026).

The computing substrate is exposed through FaaS platforms such as OSCAR+Knative or Fission on Kubernetes. Scientific workflows are decomposed into modular, containerised functions (e.g., radio interferometry routines, image convolutions) that are registered via site-wide function catalogues and exposed as authenticated REST endpoints by an API gateway (Ríos-Monje et al., 2023, Parra-Royón et al., 12 Jan 2026). The orchestration layer's Execution Planner assigns function invocations to SRC nodes where both data and idle compute resources converge; this minimises both execution time and energy consumption, formulated as a multi-objective optimisation under resource and network constraints (Parra-Royón et al., 5 Jan 2026):

$$\begin{aligned} & \min f_{1}(x) = T_{\text{exec}}(x),\quad \min f_{2}(x) = E_{\text{cons}}(x) \ & \text{subject to resource, data locality, and transfer cost constraints} \end{aligned}$$

The function invocation/dispatch process involves OIDC-based user authentication, permissions API checks, routing through GateKeeper, local pod execution with data-mount via RSEs, and streaming of results (Parra-Royón et al., 12 Jan 2026, João et al., 19 Dec 2025). This architecture enables horizontal concurrency (scaling to thousands of simultaneous function executions), elastic resource provisioning, and fine-grained cost control (Ríos-Monje et al., 2023).

3. Data Management, Distribution, and Replication Models

At the core of SRCNet's data distribution is Rucio, inherited from the WLCG community, providing a global logical namespace, robust metadata cataloguing, replicating data across RSEs via managed transfer queues (FTS), and supporting protocols such as WebDAV, xRootD, and S3 (Parra-Royón et al., 2023, Parra-Royón et al., 12 Jan 2026).

Integration of new storage endpoints (e.g., Spanish SPSRC via StoRM+WebDAV+CephFS) follows a standardised deployment and registration workflow, enabling the rapid onboarding of new nodes and interoperable authentication through OIDC (Parra-Royón et al., 2023). Performance benchmarks with multi-stream WebDAV have demonstrated aggregate transfer rates of ≥700 MiB/s over academic WAN links, with CephFS pools absorbing ~2 TiB/day of datasets in pilot deployments.

For advanced data products with multi-site generation (e.g., WALLABY HI survey), asymmetric, publisher-centric asynchronous replication strategies based on PostgreSQL+Bucardo have been deployed. This model eliminates multi-master conflicts, sustains replication rates of ≥30 MB/s per node, and achieves end-to-end latencies ~2–3 s per batch, scaling linearly with the number of federated sites (Parra-Royon et al., 2023).

4. Semantic Interoperability and Workflow Brokerage

Achieving federated orchestration, resource brokering, and global reproducibility at exascale mandates a robust, machine-actionable semantic model (João et al., 19 Dec 2025, Garrido et al., 12 Jan 2026). The SRCNet semantic model is formalised in JSON-LD, defining classes for regional centres, nodes, services, resources, and workflows. Properties encode static and dynamic resource descriptions (CPU, GPU, memory, bandwidth, availability), connectivity (isComposedBy, providesService), and workflow relationships (usesResource, dependsOn).

The formal mathematical backbone defines:

• Node set $N = \{n_i\}$, resources $R$, services $S$, workflows $W$
• Relations: allocates $(N \times R)$, provides $(N \times S)$, usesResource $(S \times R)$, executes $(N \times W)$, dependsOn $(W \times W)$
• Dynamic functions: $cpuAvail(n, t)$, $bwAvail(n, t)$, supporting real-time brokerage

The broker resolves resource assignment by querying the federated semantic endpoints (SPARQL), applying constraints and cost functions:

$$n^* = \arg\min_{n \in N} \left[ \alpha \cdot \text{latency}(n,w) + \beta \cdot \text{energyCost}(n, R_w) \right]$$

The model supports either a centralised RDF registry or a decentralised peer-to-peer (federated endpoint) configuration, facilitating both static discovery and dynamic, availability-driven planning (João et al., 19 Dec 2025).

5. FAIR Principles, Provenance, and Reproducibility

SRCNet is explicitly designed to meet the Findable, Accessible, Interoperable, and Reusable (FAIR) requirements (Possenti, 4 Apr 2025, Garrido et al., 12 Jan 2026). Each data entity—raw, intermediate, or advanced product—receives a persistent identifier, is registered with rich, standardised metadata (e.g., IVOA ObsCore), and is discoverable through distributed catalogue services.

Reproducibility is enforced via machine-captured provenance graphs based on W3C PROV-DM and its astronomical specialisations (IVOA Provenance DM, ProvONE). Every workflow step/activity is encapsulated in a container image with versioned dependencies and records mandatory provenance relationships:

• $G_{\rm prov} = (V, E)$, $$V = \mathcal{E} \cup \mathcal{A} \cup \mathcal{P} \cup \mathcal{C}$$, $E \subseteq V \times \mathcal{R} \times V$
• Containerised workflows are registered with input parameters and preserved images (Garrido et al., 12 Jan 2026, João et al., 19 Dec 2025)

Automatic capture of provenance links, validation of outputs via domain-specific quality gates, and persistent identifier assignment underpin the network’s capacity for verifiable science. A global workflow catalogue supports replay and auditability down to the container image and dataset level, with reproducibility scores tracking metadata completeness.

6. Governance, Resource Heterogeneity, and Deployment Practice

SRCNet governance is layered: the SKAO Council delegates to the SRC Steering Committee (SRCSC), which charters national and cross-institutional agile teams for prototyping, standards development, and operations (Possenti, 4 Apr 2025, 2206.13022). A resource board manages pledges of compute, storage, and staffing.

Each SRCNet node is independently funded, with architectures optimised for local heterogeneous resources (x86, ARM, GPU, NVMe/tape, low-latency fabrics), but conforming to federated protocols, containerisation, authentication, and API standards. Nodes may deploy additional site-specific services—Jupyter, CARTA, or custom workflow engines—but all present a consistent abstraction for global brokerage (Wadadekar et al., 2022, 2206.13022).

Automation of deployment is conducted via Infrastructure-as-Code (e.g., Ansible, Helm, GitOps pipelines). The Spanish SPSRC prototype demonstrated new storage element onboarding in days using automation and standard Rucio protocols (Parra-Royón et al., 2023). Network architecture leverages dual-plane LAN fabrics and multi-100Gbps peering to ensure resilience and high-throughput ingest (Wadadekar et al., 2022, 2206.13022).

7. Performance, Scalability, and Future Directions

Benchmarks and pilot deployments report substantive performance gains from computation-to-data models:

• MOEA-driven placement reduced execution times by 25–40% and energy by 15–30% versus static centralised pipelines; network traffic dropped by 50% in test cases (Parra-Royón et al., 5 Jan 2026).
• Serverless FaaS chains demonstrated end-to-end workflow speedups of up to 2.5× and up to 90% reduction in networked data transfer for multi-step pipelines (Parra-Royón et al., 12 Jan 2026).
• Database replication (WALLABY use case) delivered linear scalability and single-digit-second end-to-end lag at continental scale (Parra-Royon et al., 2023).

Scaling logic for resource allocation, storage tiering, and replication has been validated—e.g., $S(t) = S_0 + \Delta S \cdot t$ for linear growth, with provisions for transitioning to exponential scaling in simulation-intensive use cases (Wadadekar et al., 2022).

Challenges remain in sustaining elasticity and performance at 700+ PB/year, federating distributed and heterogeneous hardware, maintaining policy and access compliance, and embedding advanced energy and sustainability metrics (João et al., 19 Dec 2025, Garrido et al., 12 Jan 2026). Upcoming work targets deployment of dynamic, AI-assisted brokerages; richer, multi-objective scheduling (including monetary and reliability factors); expanded semantic integration; and full lifecycle, metadata-driven reproducibility and auditability at the SKA science scale.

---

Key References:

• (Parra-Royón et al., 5 Jan 2026) Bringing computation to the data: A MOEA-driven approach for optimising data processing in the context of the SKA and SRCNet
• (Matute-Cañadas et al., 2023) Semantic Model for the SKA Regional Centre Network
• (Parra-Royón et al., 2023) Integration of storage endpoints into a Rucio data lake, as an activity to prototype a SKA Regional Centres Network
• (Parra-Royón et al., 12 Jan 2026) Bringing Computation to the data: Interoperable serverless function execution for astrophysical data analysis in the SRCNet
• (Parra-Royon et al., 2023) Asymmetric distribution of data products from WALLABY, an SKA precursor neutral hydrogen survey
• (Possenti, 4 Apr 2025) The challenge of the data in the SRCnetwork
• (Wadadekar et al., 2022) Plans for building a prototype SKA regional centre in India
• (2206.13022) Status and progress of China SKA Regional Centre prototype
• (Garrido et al., 12 Jan 2026) Advanced computing for reproducibility of astronomy Big Data Science, with a showcase of AMIGA and the SKA Science prototype