EvoSort: A Genetic-Algorithm-Based Adaptive Parallel Sorting Framework for Large-Scale High Performance Computing

Abstract: In today's era of big data, sorting enormous datasets is a major challenge. We present EvoSort, an adaptive parallel sorting framework that employs a Genetic Algorithm (GA) to automatically discover and refine critical parameters, including insertion sort and fallback thresholds, tile size, and mergesort vs Least Significant Digit (LSD) radix sort. EvoSort integrates parallel sorting primitives and adapts continuously to input data and system architecture, ensuring optimal performance. Experiments on up to 10 billion elements show that EvoSort consistently outperforms NumPy sorting by factors from three to over 90 times. EvoSort exemplifies a powerful auto-tuning solution for large-scale data processing.