Quantum Optimization Benchmarking Library - The Intractable Decathlon

Abstract: Through recent progress in hardware development, quantum computers have advanced to the point where benchmarking of (heuristic) quantum algorithms at scale is within reach. Particularly in combinatorial optimization - where most algorithms are heuristics - it is key to empirically analyze their performance on hardware and track progress towards quantum advantage. To this extent, we present ten optimization problem classes that are difficult for existing classical algorithms and can (mostly) be linked to practically relevant applications, with the goal to enable systematic, fair, and comparable benchmarks for quantum optimization methods. Further, we introduce the Quantum Optimization Benchmarking Library (QOBLIB) where the problem instances and solution track records can be found. The individual properties of the problem classes vary in terms of objective and variable type, coefficient ranges, and density. Crucially, they all become challenging for established classical methods already at system sizes ranging from less than 100 to, at most, an order of 100,000 decision variables, allowing to approach them with today's quantum computers. We reference the results from state-of-the-art solvers for instances from all problem classes and demonstrate exemplary baseline results obtained with quantum solvers for selected problems. The baseline results illustrate a standardized form to present benchmarking solutions, which has been designed to ensure comparability of the used methods, reproducibility of the respective results, and trackability of algorithmic and hardware improvements over time. We encourage the optimization community to explore the performance of available classical or quantum algorithms and hardware platforms with the benchmarking problem instances presented in this work toward demonstrating quantum advantage in optimization.

• The paper presents QOBLIB, a framework using 10 challenging combinatorial optimization problem classes to benchmark quantum and classical algorithms.
• It employs mathematical programming formulations such as MIP and QUBO to streamline constraints and ensure numerical accuracy in problem representation.
• Experimental setups compare quantum techniques like QAOA against classical solvers, establishing a baseline for evaluating quantum advantage.

Quantum Optimization Benchmarking Library - The Intractable Decathlon

Introduction

The paper discusses the use of quantum computers for benchmarking heuristic quantum algorithms with an emphasis on combinatorial optimization. The paper presents ten classes of optimization problems that are notably challenging for classical algorithms, presenting a foundation for fair and comparable benchmarks for quantum optimization solutions. The "Quantum Optimization Benchmarking Library" (QOBLIB) is introduced to provide a structured repository for these problem instances and associated solution tracks. The goal is to enable further exploration of quantum solutions in optimization by providing systematically challenging problem instances.

Problem Classes

The ten combinatorial optimization problem classes are designed to represent practical applications and vary in complexity, coefficient ranges, and variable types. These problem types become non-trivial for classical methods at system sizes from under 100 up to approximately 100,000 decision variables.

Implementation Strategy

• Solution Representation: Optimization problems can be represented in various forms such as Mixed Integer Programming (MIP) or Quadratic Unconstrained Binary Optimization (QUBO). This paper utilizes mathematical programs for formulating optimization problems which include binary variables, simplifying constraints, and rounding coefficient values for numerical accuracy and simplicity.
• Benchmarking: The paper emphasizes the importance of benchmarking through systematically constructed problem instances that are challenging for state-of-the-art solvers. An essential aspect is providing a benchmark library that allows comparison between quantum and classical optimization solutions.

Practical Considerations

Researchers and developers deploying quantum optimization solutions can access problem instances from the QOBLIB repository. These instances are specifically crafted to challenge even the most advanced classical algorithms, making them suitable starting points for exploring quantum advantage.

Example Implementation

1. Classical Baseline: Applications utilize state-of-the-art classical algorithms such as Gurobi and ABS2 as benchmarks against which quantum optimizations can be tested.
2. Quantum Implementation: Quantum algorithms such as QAOA (Quantum Approximate Optimization Algorithm) are applied to problem instances to determine their performance and efficiency.
3. Experimental Setup: It's crucial to assess multiple factors such as computational resources, solution quality, and optimality bounds. Reporting standards including total runtime, utilized hardware, and algorithmic details are established to ensure fair comparisons.

Conclusion

The development of the QOBLIB enhances the ability to track and push forward the capabilities of quantum optimization. This framework encourages a collective effort within the scientific community to systematically explore the potential for quantum advantage, providing vital data and challenges needed for advancing optimization methodologies. This structured approach offers the chance for methodical assessment and evolution of both quantum computing technologies and the algorithms that leverage them.