Integrated Human-in-the-Loop Systems

• Integrated Human-in-the-Loop systems are frameworks that embed human decision-making within automated control loops to directly influence system behavior.
• They employ digital twins, motion capture, and multimodal sensing to continuously integrate human inputs with real-time system analytics.
• These systems provide quantifiable resilience and performance metrics, enabling comparative evaluations of expert versus novice operator responses.

Integrated Human-in-the-Loop (HITL) systems are those in which human agents are programmatically and instrumentally embedded within the operational or learning loops of cyber-physical, socio-technical, or AI-driven systems. Unlike systems that rely on isolated or post-hoc human intervention, integrated HITL architectures systematically close the loop between real human actions, physical hardware or digital simulators, automated control, and data-driven analytics, such that human influence directly shapes system trajectories, resilience metrics, and emergent system-level behaviors.

1. Architectural Principles and Formal Models

Integrated HITL frameworks instantiate closed-loop architectures where human actors and automation interact through actuators, sensors, digital twins, and interfaces. A canonical example is the Human-Hardware-in-the-Loop (HHIL) framework, which embeds real operators and hardware into the simulation of a cyber-socio-technical system (CSTS). The HHIL loop consists of multiple blocks: the physical process (plant), sensors/actuators, an industrial controller (e.g., RevPi running PID loops), SCADA/data-center interfaces, a human controller via HMI, and a digital twin. This structure enables empirical operator action traces to modulate safety-critical closed-loop dynamics (Simone et al., 8 Sep 2025).

Formally, the control dynamics in HITL systems are typically expressed as:

$$\dot{x}(t) = A x(t) + B u(t) + D d(t), \quad y(t) = C x(t)$$

where $u(t)$ includes both automated and human-generated control actions, and $d(t)$ represents external disturbances (e.g., cyber-attacks). For safety assurance in HITL-autonomous systems, synthesis frameworks such as control Lyapunov-barrier function (CLBF) controllers extend the closed-loop model:

$$\dot X = f(X) + g(X)\bigl(\pi_{\rm nom}(X) + u_{ex}(t)\bigr)$$

with human actions $u_{ex}(t)$ modeled explicitly (e.g., via Markov chains and fuzzy inference systems), and safety certification enforced by constructing barrier certificates $V(X)$ satisfying invariance and decrease conditions under all plausible human inputs (Banerjee et al., 2024).

Digital twins, motion capture, SCADA/HMI, and feedback signals serve as the communication substrate between humans, hardware, and software, enabling real-time integration of human timing, judgment, and adaptive behaviors into the system state (Simone et al., 8 Sep 2025, Chen et al., 11 Feb 2025).

2. Human Behavioral Integration and Sensing

Contemporary integrated HITL platforms employ digital twinning and multimodal sensing to empirically represent and capture operator behaviors:

• Motion capture: High-frequency inertial suits (e.g., 17-sensor, 90–240 Hz) record operator kinematics. Presence is inferred via 3D control volumes; events are injected when dwell time criteria are satisfied (Simone et al., 8 Sep 2025).
• Human-Machine Interfaces: Operators interact via HMIs, directly inputting digital setpoints or tuning controller parameters (e.g., PID gains, valve positions).
• Implicit and explicit feedback: Ratings, verbal comments, and motion-based cues are merged to drive fine-tuning in continuing learning systems (Chen et al., 11 Feb 2025).
• Raw correction factors: In mapping (e.g., HitL-SLAM), human-drawn features are interpreted as probabilistic factors through expectation-maximization, becoming new terms in pose graph optimization (Nashed et al., 2017).
• Supervision layers and overrides: Hierarchical HITL learning architectures enable human action advisory, demonstration, and direct reward shaping at several abstraction levels; buffer and sampling ratios control the strength and phase-out of human influence (Arabneydi et al., 23 Apr 2025).

Empirical human responses are timestamped and fused with the system’s digital model to generate unique execution traces that explicitly encode the effects of real operator actions under test or attack conditions.

3. Formal Safety, Resilience, and Performance Metrics

Integrated HITL systems enable new classes of quantitative and qualitative performance metrics that are unachievable in purely digital or hardware-only simulations:

• Resilience curve (area-under-curve indicator): For variable $i$,

$$A_{si} = \int_0^T y^{\mathrm{nominal}}_i(t) dt, \quad A_{di} = \int_0^T y^{\mathrm{disrupted}}_i(t) dt, \quad R_i = \frac{A_{si} - |A_{si} - A_{di}|}{A_{si}}$$

with post-alignment and masking to focus on deviations during anomaly windows (Simone et al., 8 Sep 2025).

• Operator detection and restoration times $$(\Delta t_{\mathrm{det}},\ \Delta t_{\mathrm{res}})$$: These temporal metrics, measured via motion sensors and plant state logs, quantitatively link operator performance to system resilience, revealing monotonic declines and critical thresholds beyond which recovery is poor.
• Time-to-first-warning and module latencies: In aviation safety, human-in-the-loop multimodal fusion pipelines (e.g., AIRHILT) report end-to-end reaction latencies (ASR, vision, TTS) and time-to-warning metrics to benchmark pilot/controller assistance (Garib et al., 24 Nov 2025).
• Empirical evaluation: Monte Carlo integration over operator traces, attack scenarios, and plant time-series generates quantitative distributions of resilience metrics under expert vs. novice behavior (Simone et al., 8 Sep 2025).

4. Comparison: Automation, Human Expertise, and Hybridization

Integrated HITL systems make it possible to rigorously compare and combine the benefits of human expertise and automated control:

Mode	Human Role	System Outcomes
Pure Automation (no human loop)	None or "observer"	High consistency, deterministic; rigid
Standalone Human-in-the-Loop (episodic)	Sparse correction/manual patch	Corrects major errors; not continuous
Integrated HITL (closed loop)	Empirically-captured, continuous	Quantifiable resilience, adaptation, operator-dependent trade-offs

Empirical case studies highlight that experts achieve faster detection and higher resilience than novices, but at the cost of occasional “too rigid” routines. Novices may, in some scenarios, exceed expert performance via improvisational adaptation if failures are sufficiently non-critical. Monotonic resilience deterioration as a function of delayed detection or restoration is consistently observed, as are critical thresholds where resilience collapses (Simone et al., 8 Sep 2025).

Such findings would not be possible in the absence of direct empirical integration of physical humans and hardware into the simulation and evaluation pipeline.

5. System Generalizability, Assumptions, and Deployment Challenges

Integrated HITL frameworks rest on several operational and modeling assumptions:

• Testbed fidelity: Requires representative physical/simulated testbeds and accurate digital twins. Cost and hazard constraints may limit deployment in large-scale or dangerous industries.
• Behavioral scope: Current implementations may restrict human behavioral modeling to a subset of signals (e.g., motion capture, HMI logs) and typically do not capture higher-cognitive, stress, or coordinated team interactions.
• Attack surface: Most research focuses on specific cyber-attack classes (e.g., sensor FDI/DoS); richer or multi-modal adversarial scenarios necessitate extensions to the modeled control action/feedback spaces.
• Metric scope: The “resilience triangle” metric is a pragmatic univariate proxy; multi-dimensional adaptation, recovery re-planning, or networked cascade effects remain open for future HITL metric frameworks.

For cross-domain portability, minimum requirements are: a physical or high-fidelity simulated plant, STAMP/STPA-based safety control structure design, human instrumentation (digital twin, wearables), integrated SCADA/HMI to close the loop, and domain-appropriate resilience or safety metrics (Simone et al., 8 Sep 2025).

6. Impact, Design Insights, and Future Directions

The empirical and methodological advances enabled by integrated HITL architectures yield several profound outcomes:

• Bridging qualitative and quantitative gaps: Real human timing and decision dynamics can be measured, analyzed, and directly linked to system-level resilience and safety under complex perturbations.
• Hybrid system design: Provides structured pathways for engineering CSTS that leverage the advantages of both automation and rapid, adaptive human cognition.
• Revealing trade-offs: Quantifies expert consistency vs. novice adaptability, the dominant influence of time-to-detection, and the interplay of operator routine rigidity and scenario criticality.
• Path to future work: Demands extensions to multi-faceted resilience metrics, richer behavior/cognition models, scalable and safe testbed architectures, and increased coverage of heterogeneous cyber-physical adversaries.

Integrated HITL thus represents a foundational framework for quantitatively engineering, analyzing, and improving resilience and safety in the next generation of complex socio-technical systems, with direct applicability to industrial, infrastructural, and cyber-physical domains (Simone et al., 8 Sep 2025).