Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics

Abstract: Modern LLM serving operates in highly volatile environments characterized by severe runtime dynamics, such as workload fluctuations and elastic cluster autoscaling. Traditional serving systems rely on static, human-engineered serving policies (e.g., scheduling algorithms and rescheduling strategies) to manage these dynamics. However, these policies must navigate deeply intertwined runtime trade-offs (e.g., scheduling overhead vs. execution efficiency, rescheduling frequency vs. reconfiguration overhead), whose optimal balance is workload-specific and shifts continuously as runtime conditions evolve, rendering any fixed policy fundamentally unable to adapt. We propose Autopoiesis, a novel online self-evolving system that shifts LLM serving from static policy deployment to continuous online policy evolution. First, Autopoiesis introduces an LLM-driven program synthesis workflow to evolve serving policies with respect to real-time observed dynamics, where the evolved policies reflect the optimal decision in navigating the complex, multi-dimensional trade-off space. Second, Autopoiesis enables this synthesis process to operate continuously during serving, observing real-world system behavior, and rewriting the policy code as runtime trade-offs shift, thereby transforming policy design from a one-time offline endeavor into an ongoing system component, enabling autonomous adaptation to evolving runtime conditions. Together, we establish a new paradigm: Serving policies are no longer static artifacts designed by humans before deployment, but living code that LLMs continuously evolve throughout deployment to navigate runtime trade-offs beyond human design. We evaluate Autopoiesis across diverse runtime dynamics and show up to 53% and on average 34% improvements over state-of-the-art LLM serving systems.

• The paper introduces a self-evolving system paradigm that generates serving policies via LLM-driven evolutionary program synthesis.
• It decouples policy execution and evolution with a two-plane architecture that optimizes serving under dynamic workload and hardware conditions.
• Empirical results show up to a 53% reduction in execution time and 13x faster scheduling compared to static, human-engineered policies.

Autopoiesis: Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics

Motivation and Limitations of Static Serving Policies

LLM serving operates in production environments marked by highly dynamic workloads and elastic cloud clusters. Traditional systems utilize static, human-engineered serving policies—including fixed scheduling algorithms, rescheduling triggers, and reconfiguration heuristics—to manage these runtime fluctuations. Such policies fundamentally struggle to adapt to the deeply intertwined trade-offs of scheduling, rescheduling, and reconfiguration, whose optimal balance is constantly shifting. Static approaches fail to navigate this multi-dimensional optimization space effectively: even finely-tuned policies become suboptimal as runtime dynamics diverge from preconfigured expectations, resulting in degraded throughput and increased latency.

Figure 1: Impact of workload dynamics on serving policy performance: Static policies collapse under bursty or steady shifts, while adaptive policies maintain throughput.

Paradigm Shift: Continuous Online Policy Evolution

Autopoiesis introduces a novel system paradigm: serving policies are no longer static artifacts but continuously evolving programs, synthesized online via LLM-driven evolutionary workflows. The system decouples policy execution from policy evolution using a two-plane architecture. The data plane executes the active serving policy, dynamically generating serving plans and periodically snapshotting recent runtime traces. The control plane asynchronously evolves policy code using LLM-driven program synthesis, optimizing for end-to-end serving efficiency given real-time workload and cluster conditions. When superior policies are discovered, they are hot-swapped into the data plane, ensuring the serving logic adapts proactively to changing operational realities.

Figure 2: Overview of Autopoiesis’s two-plane architecture: Data plane executes policies while control plane evolves policies in a self-adaptive loop.

LLM-Driven Program Synthesis for Serving Policy Optimization

The central mechanism enabling Autopoiesis is an LLM-driven evolutionary program synthesis workflow. Policy optimization is formulated as minimizing total trace completion time under continuous execution constraints, where scheduling and reconfiguration decisions impact concurrent serving throughput. The policy interface consists of two co-evolved functions: $should\_reschedule(ctx)$, which determines rescheduling points, and $schedule(ctx)$, which generates new serving plans. These functions jointly navigate trade-offs between rescheduling frequency, scheduling thoroughness, and reconfiguration aggressiveness.

The synthesis workflow initializes policy populations with diverse algorithmic seeds, mutates policy code via LLMs guided by per-iteration artifact feedback (structured cost breakdowns from trace-replay evaluators), and selects elite candidates via MAP-Elites-inspired mechanisms. Candidate scoring is based on realistic trace replay, using a calibrated serving simulator for throughput and transition overhead estimation. This allows targeted exploitation of workload- and hardware-specific trade-offs.

Figure 3: LLM-driven program synthesis workflow: Evaluator produces fitness feedback, guiding mutation and selection for optimal policy evolution.

System Implementation and Efficient Evolution

Efficiency is ensured through warm-start re-evolution, leveraging prior evolved policies as initialization for new snapshots. Multi-level timeouts bound candidate and cycle runtimes to prevent divergences from LLM hallucinations. Asynchronous policy hot-swap enables immediate deployment of evolved policies without disrupting live serving. Sliding-window snapshotting provides up-to-date system state for optimal policy adaptation.

Figure 4: Policy hot-swap and trace snapshotting: New policies are asynchronously deployed as snapshots inform continuous evolution.

Empirical Evaluation: Quantitative Performance and Qualitative Adaptation

Autopoiesis consistently outperforms state-of-the-art serving systems (DistServe, HexGen, SpotServe) across homogeneous, heterogeneous, and elastic cloud clusters by substantial margins. End-to-end execution time reductions reach up to 53% (average 34-44%), driven by workload-adaptive minimization of scheduling and reconfiguration overhead during volatile periods and proactive plan optimization during stable workload regimes. Evolved policies autonomously discover architectural strategies tailored to hardware heterogeneity (e.g., bounding tensor parallelism within intra-machine, prioritizing high-performance GPUs for large models, scenario-specific workload assignment granularity). In agentic workflow scheduling and online request assignment, evolved schedulers reduce scheduling time by up to 13x and improve completion time over greedy and MILP baselines without manual tuning.

Figure 5: End-to-end execution time comparison: Autopoiesis achieves superior performance across all tested clusters.

Figure 6: Evolution process convergence: Rapid discovery of high-quality policies in both stable and volatile workload periods.

Figure 7: Warm-start re-evolution sharply reduces evolution time for consecutive snapshots, optimizing adaptation efficiency.

Analytical Insights: Navigating Deep Trade-Offs

Controlled case studies illustrate that neither greedy nor ILP-based static policies universally dominate; optimal trade-off navigation is workload regime dependent. Autopoiesis's synthesis workflow reshapes the Pareto frontier, adapting both the scheduling algorithm's computational profile and the rescheduling strategy to workload volatility. Similar advantages are realized in elastic cluster scenarios: evolved reconfiguration strategies minimize unnecessary transitions, capturing serving gains with moderate overhead—a behavior impenetrable to static full- or minimal-migration policies.

Figure 1: Dynamic adaptation of policy trade-offs across volatile and steady phases: Evolved policies sustain throughput by continuously navigating serving, scheduling, and reconfiguration costs.

Practical and Theoretical Implications

Autopoiesis establishes a generalizable paradigm for system control under evolving operational landscapes. Theoretical implications extend to automated discovery of domain-specific scheduling principles, opening new directions for program synthesis in combinatorial system settings. Practically, it minimizes operational cost and latency, enabling production-grade LLM serving to reactively and proactively adapt at sub-policy granularity. Warm-start evolution and parallel candidate evaluation yield operational scalability, and model-agnostic synthesis constructs extend to alternative optimization objectives (e.g., SLO attainment, percentile latency minimization). Deployment considerations include LLM API stability, adaptive snapshot window sizing, and evolutionary budget management.

Conclusion

Autopoiesis fundamentally shifts LLM serving system design from static, human-engineered policies to living code evolved online via LLM-driven program synthesis. Through a decoupled two-plane architecture and trace-replay-based evaluators, the system continuously adapts to shifting runtime trade-offs, achieving robust throughput and efficiency improvements across diverse serving regimes. This paradigm significantly advances autonomous system control, with implications for both practical deployment and algorithmic discovery in dynamic environments.