It's the People, Not the Placement: Rethinking Allocations in Post-Moore Clouds

Abstract: The Cambrian explosion of new accelerators, driven by the slowdown of Moore's Law, has created significant resource management challenges for modern IaaS clouds. Unlike the homogeneous datacenters backing legacy clouds, emerging neoclouds amass a diverse portfolio of heterogeneous hardware -- NVIDIA GPUs, TPUs, Trainium chips, and FPGAs. Neocloud operators and tenants must transition from managing a single large pool of computational resources to navigating a set of highly fragmented and constrained pools. We argue that cloud resource management mechanisms and interfaces require a fundamental rethink to enable efficient and economical neoclouds. Specifically we propose shifting from long-term static resource allocation with fixed-pricing to dynamic allocation with continuous, multilateral cost re-negotatiaton. We demonstrate this approach is not only feasible for modern applications but also significantly improves resource efficiency and reduces costs. Finally, we propose a new architecture for the interaction between operators, tenants, and applications in neoclouds.