Quantifying Energy-Efficient Edge Intelligence: Inference-time Scaling Laws for Heterogeneous Computing

Abstract: LLM inference on resource constrained edge devices remains a major challenge for low latency intelligent systems, as existing solutions depend heavily on cloud or datacenter infrastructure. This work introduces QEIL, Quantifying Edge Intelligence via Inference time Scaling Laws, a unified framework for efficient local LLM inference using principled scaling laws and heterogeneous orchestration across CPU, GPU, and NPU accelerators. We derive five architecture agnostic theorems that characterize how inference efficiency scales with model size, sample budget, and device level constraints. QEIL integrates three optimization dimensions. First, inference time scaling laws show that heterogeneous workload distribution achieves superlinear efficiency gains that are not observed in homogeneous execution. Second, hardware aware routing is enabled through analytical cost models that account for compute throughput, memory bandwidth, power consumption, and thermal limits. Third, performance energy trade offs are quantified using novel metrics including Intelligence Per Watt, Energy Coverage Efficiency, and Price Power Performance. A unified orchestrator combines these components through progressive sample multiplexing to improve coverage. Extensive evaluation across five model families from 125M to 2.6B parameters demonstrates consistent gains, including 7 to 10.5 percentage point improvement in pass at k coverage, 35.6 to 78.2 percent energy reduction, 68 percent average power reduction enabling edge thermal budgets, 15.8 percent latency improvement, and zero accuracy loss. Results confirm that inference time scaling laws are universal and architecture agnostic, establishing heterogeneous edge orchestration as the optimal strategy for energy constrained intelligent systems.