Benchmarking the performance of quantum computing software

Abstract: We present Benchpress, a benchmarking suite for evaluating the performance and range of functionality of multiple quantum computing software development kits. This suite consists of a collection of over $1000$ tests measuring key performance metrics for a wide variety of operations on quantum circuits comprised of up to $930$ qubits and $\mathcal{O}(10^{6})$ two-qubit gates, as well as an execution framework for running the tests over multiple quantum software packages in a unified manner. We give a detailed overview of the benchmark suite, its methodology, and generate representative results over seven different quantum software packages. The flexibility of the Benchpress framework allows for benchmarking that not only keeps pace with quantum hardware improvements but can preemptively gauge the quantum circuit processing costs of future device architectures. Being open-source, Benchpress ensures the transparency and verification of performance claims.