Workload-Guided Execution Time Estimator (ETE)

• Workload-guided ETEs are models that incorporate empirical features like input characteristics and hardware state to improve runtime predictions over conservative static analyses.
• They use techniques such as dynamic instrumentation, feature selection, and machine learning regression to construct accurate and adaptive execution time models.
• ETE outputs inform proactive scheduling and resource management, achieving near 3% error margins and significant performance gains in real-world applications.

A Workload-Guided Execution Time Estimator (ETE) is a class of models, algorithms, and tooling for predicting or bounding software execution time, where parameters or features derived from the actual workload—such as program input characteristics, hardware state, or empirical runtime behavior—play a central role in model construction, calibration, or application. ETEs contrast with conventional static resource analysis by augmenting or replacing conservative, program- or architecture-wide assumptions with predictions or measurements contextualized to the target workload. These methods have been instantiated in compiler analysis frameworks, machine learning-driven scheduling systems, measurement-profiling pipelines, and probabilistic real-time systems modeling.

1. Workload-Guided ETE: General Principles and Motivation

Workload-guided ETEs depart from architecture- or code-agnostic timing models by injecting empirical or simulated characteristics of real-world executions into their estimation process. This may take the form of selecting predictive features from code and input data, measuring representative execution paths, or mining sequence recurrences from deployed workloads. Principal motivations include:

• Reducing pessimism of worst-case execution time (WCET) by focusing modeling capacity on actually observed paths and patterns (Stattelmann et al., 2014, Bielmeier et al., 30 Jul 2025).
• Improving early- or compile-time predictability for new or structured workloads (loops, ML pipelines, microservice batches) (Xu et al., 17 Mar 2025, Mururu et al., 2021, Kumar et al., 10 Sep 2025).
• Facilitating adaptive or mixed-mode analysis by incorporating runtime feedback, e.g., via tracepoints or beacon events (Bielmeier et al., 30 Jul 2025, Mururu et al., 2021).
• Accommodating heterogeneous hardware and input diversity by parameterizing over workload-specific features (Kumar et al., 10 Sep 2025, Fan et al., 26 Dec 2025).

2. Approaches: Instrumentation, Feature Selection, and Profiling

ETE deployments employ a variety of workload-guided strategies for data acquisition and model feature construction.

Instrumentation and tracing:

Instrumentation can occur statically (in the compiler front-end), dynamically (at load/run time), or via hybrid techniques (e.g., timed tracepoints in kernels and userspace) (Xu et al., 17 Mar 2025, Mururu et al., 2021, Bielmeier et al., 30 Jul 2025). Features often include:

• Instruction, memory, and control-flow counts (e.g., IR instructions, branch/jump behavior).
• Cache or branch-predictor miss statistics through simulated or hardware-logged events.
• Input-dependent features (file size, input shape, entropy, sequence quality for genomics) (Kumar et al., 10 Sep 2025).

Profiling and basis selection:

Measurement-based ETEs select representative or combinatorially sufficient paths or input configurations for timing, often via systematic path enumeration, barycentric spanners, or iterative LP/ILP procedures (Bundala et al., 2015). For codebases with high code sequence recurrence, models extract and profile instruction windows weighted by occurrence counts (Stattelmann et al., 2014).

Feature vector construction:

Feature sets can be rich, including static-symbolic loop bounds, access footprints, reuse distance, or full high-dimensional vectors logging program dynamic events (Xu et al., 17 Mar 2025, Mururu et al., 2021). In domain pipelines (e.g., LLM inference, genomics), features typically mix input quantities (size, shape) and domain-specific statistics (GC-content, cache occupancy) (Fan et al., 26 Dec 2025, Kumar et al., 10 Sep 2025).

3. Modeling Techniques: Analytical, ML, and Probabilistic Methods

ETE models range from analytical function fitting to machine learning regressors and probabilistic generative models. Key modeling patterns include:

Regression/fitting:

• Analytical forms, such as step-structured linear or polynomial models for structured workloads (LLM decoding, nested loops) (Fan et al., 26 Dec 2025, Mururu et al., 2021).
• Linear and non-linear regression (ordinary least squares, regularized, Huber, decision trees, random forests, neural networks) for mapping feature vectors to runtimes (Xu et al., 17 Mar 2025, Kumar et al., 10 Sep 2025, Mururu et al., 2021).

Symbolic/ILP parametric models:

• Edge-parameterized execution models, learned via measurement consistency LPs and solved for per-edge weights and platform jitter, support error-bounded prediction for arbitrary input-induced paths (Bundala et al., 2015).

Probabilistic semi-Markov models:

• Workload-derived semi-Markov chains, where transitions and sojourn times are directly inferred from event traces, yielding full latency distributions via time-to-absorption analysis (Bielmeier et al., 30 Jul 2025).

Recurrence-based aggregation:

• Hashing of recurring instruction sequences in industrial binaries, with timing characterizations of canonical representatives and immediate mapping of new code to previously observed sequence digests (Stattelmann et al., 2014).

4. Integration into Systems and Scheduling Frameworks

ETE outputs drive decision processes in schedulers, resource managers, and IDE tooling.

Proactive scheduling:

• Compiler-inserted "beacons" transmit predicted phase timing and resource requirements, with the scheduler aggregating across workload classes and modes (cache reuse vs. streaming bandwidth) (Mururu et al., 2021).
• ML-driven makespan estimation enables optimal assignment in flexible job-shop pipelines; orchestration plans are generated by constraint solvers incorporating stage-wise ETE predictions (Kumar et al., 10 Sep 2025).

IDE/interactive integration:

• Recurrence-timing models provide per-code-line best/average/worst-case annotations in control application development environments. User-specified constraints (e.g., atypical paths) modulate the typical-case prediction semantics (Stattelmann et al., 2014).

Real-time inference control:

• LLM inference systems use polynomial ETEs with dynamic parameterization (prompt length, predicted response, cache eviction) to select on-the-fly strategies (e.g., aggressive KV eviction) that maximize utility while respecting a time budget (Fan et al., 26 Dec 2025).

5. Evaluation Metrics and Empirical Results

ETE methods are typically evaluated on benchmarks representative of actual workloads, with accuracy measured by absolute/symmetric percentage error, regression (R², MSE, MAE), and empirical/worst-case over/under-approximation.

Empirical findings (examples):

• PrETi RF model achieves 11.98% APE on CATREEN benchmarks, surpassing state-of-the-art by 5.4 percentage points (Xu et al., 17 Mar 2025).
• Semi-Markov ETE achieves mean-predicted WCET within ~3% of empirical maximum on cyclictest, and 99.99% quantiles stabilize after a few seconds of workload data (Bielmeier et al., 30 Jul 2025).
• In genomics pipelines, RF ETEs explain ≈90% of runtime variance with MAE ≈65 s per sample (multi-stage), enabling 2× makespan speedup in optimal scheduling vs. greedy ML baselines (Kumar et al., 10 Sep 2025).
• Recurrence-based models produce best-case, average-case, and worst-case estimates with pessimism ratios for WCET in the 1.05–1.20 range (5–20% over-approximation vs. 30–50% baseline) (Stattelmann et al., 2014).

6. Workload Focus: Adaptivity, Limitations, and Future Trends

Workload-guided ETEs adapt model structure and parameters to workload diversity and underlying system behavior.

• Trace-driven and basis-path models automatically reconfigure to new execution patterns via observed transitions and measured path coverage.
• Once models are constructed, updating for new hardware or software variants may require retraining or incremental profiling but can leverage overlap with previously observed code (Stattelmann et al., 2014, Kumar et al., 10 Sep 2025).
• Major limitations include incomplete microarchitecture coverage (e.g., pipeline stalls in PrETi (Xu et al., 17 Mar 2025)), modeling gaps for unprofiled library interactions, and limited generalization to entirely novel code patterns or resource dynamics.
• Prospective improvements focus on integrating richer microarchitectural simulation, sequential ML models for code/trace embeddings, direct support for library and OS event modeling, and tighter closed-form error bounding.

ETE methodologies have demonstrated robust performance and significant operational benefits in early design analysis, safety-critical real-time control, scalable scientific workloads, and time-sensitive inference, with a recurring theme: by learning from and adapting to the workload, execution time estimation becomes more accurate, actionable, and practical at scale.