Towards Co-execution on Commodity Heterogeneous Systems: Optimizations for Time-Constrained Scenarios

Abstract: Heterogeneous systems are present from powerful supercomputers, to mobile devices, including desktop computers, thanks to their excellent performance and energy consumption. The ubiquity of these architectures in both desktop systems and medium-sized service servers allow enough variability to exploit a wide range of problems, such as multimedia workloads, video encoding, image filtering and inference in machine learning. Due to the heterogeneity, some efforts have been done to reduce the programming effort and preserve performance portability, but these systems include a set of challenges. The context in which applications offload the workload along with the management overheads introduced when doing co-execution, penalize the performance gains under time-constrained scenarios. Therefore, this paper proposes optimizations for the EngineCL runtime to reduce the penalization when co-executing in commodity systems, as well as algorithmic improvements when load balancing. An exhaustive experimental evaluation is performed, showing optimization improvements of 7.5\% and 17.4\% for binary and ROI-based offloading modes, respectively. Thanks to all the optimizations, the new load balancing algorithm is always the most efficient scheduling configuration, achieving an average efficiency of 0.84 under a pessimistic scenario.