cuPDLPx: A Further Enhanced GPU-Based First-Order Solver for Linear Programming

Abstract: We introduce cuPDLPx, a further enhanced GPU-based first-order solver for linear programming. Building on the recently developed restarted Halpern PDHG for LP, cuPDLPx incorporates a number of new techniques, including a new restart criterion and a PID-controlled primal weight update. These improvements are carefully tailored for GPU architectures and deliver substantial computational gains. Across benchmark datasets, cuPDLPx achieves 2.5x-5x speedups on MIPLIB LP relaxations and 3x-6.8x on Mittelmann's benchmark set, with particularly strong improvements in high-accuracy and presolve-enabled settings. The solver is publicly available at https://github.com/MIT-Lu-Lab/cuPDLPx.

• The paper presents cuPDLPx, integrating a novel rHPDHG framework with adaptive restart mechanisms to significantly boost GPU-based LP solver performance.
• It employs reflected updates and PID-controlled primal weight adjustments to achieve empirical speedups up to 4.63x in LP benchmarks.
• The approach leverages GPU parallelism and recent theoretical advances, ensuring rapid convergence and stability in first-order linear programming methods.

"cuPDLPx: A Further Enhanced GPU-Based First-Order Solver for Linear Programming" (2507.14051)

Introduction to Linear Programming and Traditional Methods

Linear Programming (LP) remains a cornerstone in mathematical optimization due to its elegance and ubiquitous application across numerous domains. Traditionally, the simplex method and interior-point methods (IPMs) have dominated LP solution methodologies. The simplex method, conceptualized by Dantzig in the 1940s, effectively navigates the vertex space of feasible regions, offering crucial attainments in practical scenarios. Despite its exponential worst-case complexity, the simplex method's robustness and interpretability make it indispensable in commercial solvers. Conversely, IPMs, germinated from Karmarkar's work in the 1980s, operate within the feasible region's interior, adhering to a central path toward optimality with polynomial-time guarantees. This synergistic combination of simplex and IPM approaches constitutes the backbone of state-of-the-art CPU-based LP solvers, conferring high-accuracy solutions across diverse instance types.

Evolution of GPU-Based Solvers

Recent trends have pivoted towards leveraging Graphics Processing Units (GPUs) to accelerate LP solvers, particularly employing First-Order Methods (FOMs) such as primal-dual hybrid gradient (PDHG). GPUs' massive parallelism and memory bandwidth make them exceptionally suited for FOMs, which primarily rely on sparse matrix-vector multiplications. The burgeoning size of LP instances, potentially involving billions of variables, underscores the necessity for GPU-accelerated performance improvements. Among these advancements, cuPDLP and its successor cuPDLP-C have demonstrated notable speedups by aligning algorithmic designs with GPU architecture, influencing commercial solver designs from industry leaders like Gurobi, FICO, and NVIDIA.

cuPDLP+: Innovations and Enhancements

The paper introduces cuPDLP+, an iteration upon cuPDLP, integrating several enhancements driven by contemporary theoretical advances. Notably, cuPDLP+ adopts the restarted Halpern PDHG (rHPDHG) configuration, incorporating reflected updates and PID-controlled adjustments to dynamically update primal weights, optimizing solver performance on GPU architectures. Key enhancements include:

1. Algorithmic Refinements: Transition from raPDHG to the rHPDHG framework permits more aggressive optimization steps, bolstering empirical performance and highlighting the practical benefits derived from recent theoretical insights.
2. Adaptive Restart Mechanisms: cuPDLP+ embraces adaptive restart strategies, governed by fixed-point error metrics and aligned with theoretical guarantees to stimulate faster convergence, particularly targeting high-accuracy solutions.
3. Steady-State Optimization: Utilization of constant step-size calculations counterbalances adaptive adjustments, significantly enhancing solution stability.
4. Advanced Primal Weight Adjustments: Application of PID control regimes facilitates nuanced primal weight regulation, fostering equilibrium between primal-dual progress vectors.

Numerical Results and Performance Analysis

Empirical validations substantiated cuPDLP+'s formidable computational improvements, achieving 2x to 4x speedups over cuPDLP across LP relaxation benchmarks reliant on MIPLIB 2017 instances. Especially, high-accuracy contexts with presolve manifested up to 4.63x acceleration, underscoring cuPDLP+'s prowess in handling challenging LP scenarios with efficiency and precision. Through comprehensive benchmarks, the enhancements of cuPDLP+ demonstrate robustness in small to medium-scale LP instances, although large-scale problem benefits are comparatively moderated due to intrinsic variance.

Conclusion

The introduction of cuPDLP+ heralds significant strides in GPU-based LP solver development, amalgamating algorithmic sophistication with hardware compatibilities. As LP problem sizes magnify, the relevance of GPU solutions fortified by innovations such as reflected Halpern PDHG and adaptive strategies is increasingly pivotal. Future endeavors in AI may further harness such advancements, extending their applicability beyond existing paradigms in optimal transport, semidefinite programming, and other expansive optimization realms.