Global Hash Tables Strike Back! An Analysis of Parallel GROUP BY Aggregation

Abstract: Efficiently computing group aggregations (i.e., GROUP BY) on modern many-core architectures is critical for analytic database systems. Today's engines predominately use a partitioned approach to group aggregation, in which an incoming data stream is partitioned by key values so that every row for a particular key is sent to the same thread. In this paper, we revisit a simpler strategy: a fully concurrent group aggregation technique using a shared global hash table. While approaches using general-purpose concurrent hash tables have generally been found to perform worse than partitioning-based approaches, we argue that the key ingredient is customizing the concurrent hash table for the specific task of group aggregation. Through extensive experiments on synthetic workloads (varying key cardinality, skew, and thread counts), we demonstrate that a purpose-built concurrent hash table can match or surpass partitioning-based techniques. We also analyze the operational characteristics of both techniques, including resizing costs and memory pressure. In the process, we derive practical guidelines for database implementers. Overall, our analysis indicates that fully concurrent group aggregation is a viable alternative to partitioning.