Mitigating context switching in densely packed Linux clusters with Latency-Aware Group Scheduling

Abstract: Cluster orchestrators such as Kubernetes depend on accurate estimates of node capacity and job requirements. Inaccuracies in either lead to poor placement decisions and degraded cluster performance. In this paper, we show that in densely packed workloads, such as serverless applications, CPU context switching overheads can become so significant that a node's performance is severely degraded, even when the orchestrator placement is theoretically sound. In practice this issue is typically mitigated by over-provisioning the cluster, leading to wasted resources. We show that these context switching overhead arise from both an increase in the average cost of an individual context switch and a higher rate of context switching, which together amplify overhead multiplicatively when managing large numbers of concurrent cgroups, Linux's group scheduling mechanism for managing multi-threaded colocated workloads. We propose and evaluate modifications to the standard Linux kernel scheduler that mitigate these effects, achieving the same effective performance with a 28% smaller cluster size. The key insight behind our approach is to prioritise task completion over low-level per-task fairness, enabling the scheduler to drain contended CPU run queues more rapidly and thereby reduce time spent on context switching.