Selected Basis Diagonalization (SBD) Overview

• Selected Basis Diagonalization is a method that restricts the diagonalization process to physically or statistically important subspaces, reducing computational cost while maintaining variational bounds.
• The approach leverages quantum sampling, adaptive basis selection, and fast iterative algorithms to accurately compute energies and eigenstates in complex quantum systems.
• Applications span quantum chemistry, many-body physics, and matrix analysis, with modern implementations utilizing hybrid quantum-classical methods and GPU acceleration.

Selected Basis Diagonalization (SBD) is a computational paradigm for extracting spectral properties of large quantum systems, exploiting the restriction of the Hamiltonian to carefully selected subspaces. SBD underlies multiple classes of quantum, classical, and hybrid algorithms, notably in quantum chemistry, quantum many-body physics, matrix block decomposition, and operator theory. By focusing diagonalization efforts on subspaces generated by physically and/or statistically important basis configurations—rather than full Hilbert space—the method achieves significant reductions in computational cost, while maintaining variational bounds and well-characterized errors.

1. Theoretical Framework and Variational Properties

SBD formalizes the projection of a Hamiltonian $H$ onto a subspace $\mathcal{S}$ spanned by selected basis vectors $\{\vert x\rangle\}$, yielding a reduced matrix $$H_\mathcal{S}=\left[\langle x \vert H \vert y \rangle\right]_{x,y\in\mathcal{S}}$$. The selection of $\mathcal{S}$ is guided by physical significance (e.g., large wavefunction amplitudes), statistical sampling, or operator-theoretic criteria.

The resulting eigenproblem,

$$H_\mathcal{S} \mathbf{c}^{(k)} = E_k^{(\mathcal{S})} \mathbf{c}^{(k)}, \quad k=0,1,\ldots$$

produces approximate energies $E_k^{(\mathcal{S})}$ and eigenstates $$\vert \Phi_k \rangle = \sum_{x\in \mathcal{S}} c_x^{(k)} \vert x\rangle$$. The lowest eigenvalue satisfies the strict variational bound $E_0^{(\mathrm{exact})} \leq E_0^{(\mathcal{S})}$ as $H_\mathcal{S}$ is Hermitian and $\mathcal{S}$ a subspace, ensuring that improvements follow systematically from subspace enlargement (Kanno et al., 2023, Cantori et al., 18 Aug 2025).

2. Algorithmic Realizations Across Domains

Quantum Chemistry & Many-Body Systems

Quantum chemistry applications center around second-quantized fermionic Hamiltonians, with SBD employed to approximate ground and excited state energies via truncated determinant subspaces. A typical workflow—exemplified by quantum-selected configuration interaction (QSCI)—includes quantum or classical state preparation, sampling configurations, and subsequent classical diagonalization in the selected subspace. This approach capitalizes on quantum sampling to navigate the intractable Hilbert space, followed by classical solution of a manageable eigenproblem (Kanno et al., 2023).

Adaptive Sampling Configuration Interaction (ASCI) utilizes fast sorting and dynamic masking innovations to scale SBD to millions of determinants. Determinant selection and connectivity exploitation (single/double excitations, residue arrays) minimize redundant computation while supporting deterministic perturbation theory via sort/hash techniques (Tubman et al., 2018).

Quantum-Classical Hybrid and GPU-Accelerated SBD

Sample-Based Quantum Diagonalization (SQD) exploits hybrid resources: quantum processors select basis determinants, while scalable classical (often GPU-accelerated) engines diagonalize the projected Hamiltonian. Davidson or Lanczos iterative methods are preferred due to memory and performance constraints for large $N$, with GPU-based sparse matvec implementations benefiting from data-parallel primitives (e.g., Thrust or OpenMP offload), yielding 40–100× runtime improvements and supporting basis sizes $\sim10^9$ (Doi et al., 23 Jan 2026, Walkup et al., 22 Jan 2026).

Operator Theory and Restricted Diagonalization

On infinite-dimensional Hilbert spaces, SBD—termed restricted diagonalization—addresses the existence of unitaries close to identity (relative to an operator ideal $\mathcal{J}$) that diagonalize $A$. Necessary and sufficient conditions are expressed in terms of operator-series relations involving spectral projections and diagonal projections, encoding ideal-theoretic constraints and essential codimension obstructions (Chiumiento et al., 2021).

Simultaneous Block Diagonalization (SBD) for Matrix Sets

In network theory and matrix analysis, SBD denotes (simultaneous) block diagonalization of multiple symmetric matrices. Existence follows from matrix $\ast$-algebra structural theorems: all matrices generated by a finite set can be jointly block diagonalized. The practical algorithm involves finding the commutant via Kronecker representations, followed by eigendecomposition. Performance indices ($I_d$, $I_{cs}$) assess success—often SBD reduces random network problems only trivially, unless underlying symmetry or modularity exists (Panahi et al., 2021).

3. Basis Selection Schemes, Error Controls, and Extensions

Basis selection typically leverages:

• Quantum sampling: Importance assigned per sampling frequency $$f_x \approx |\langle x \vert \Psi_{\mathrm{trial}} \rangle|^2$$, selecting determinants with $f_x \geq \epsilon$ or top $R$ (Kanno et al., 2023).
• Neural-network sampling: Autoregressive models or neural quantum states optimize determinant selection, with adaptive basis rotation (via parameterized unitaries) focusing ground-state support even under strong delocalization (Cantori et al., 18 Aug 2025).
• Overlap truncation: In integrable systems (Bethe Ansatz), subspaces are ordered by squared overlaps with the initial state, providing numerically efficient coverage (Burovski et al., 2021).

Truncation errors obey

$$E_0^{(\mathcal{S})} - E_0^{(\mathrm{exact})} \to 0 \quad \text{ as } |\mathcal{S}| \to \dim(\mathcal{H}),$$

and sampling noise scales as $\mathcal{O}(1/\sqrt{N_{\mathrm{shots}}})$, with robust behavior especially when CI amplitudes decay rapidly (Kanno et al., 2023, Cantori et al., 18 Aug 2025). Post-selection or filtering on conserved quantum numbers further suppresses spurious contributions.

Extensions of SBD include property evaluation for arbitrary operators expressible in the computational basis (with no quantum cost once the subspace is fixed), excited-state computation (via single or sequential diagonalization schemes), and generalization to non-integrable models or operator ideals (Kanno et al., 2023, Burovski et al., 2021, Chiumiento et al., 2021).

4. Algorithmic Innovations and Computational Scaling

For large-scale classical diagonalization, algorithmic innovations include:

• Dynamic bit masking and residue arrays: Efficient construction of connectivity graphs for determinants, reducing memory and search costs.
• Fast matrix element evaluation: Exploiting local excitation structure for rapid diagonal/hopping-term computation.
• GPU acceleration: Data-parallel primitives (Thrust, OpenMP offload) coalesce determinant and excitation information for high throughput. All critical-path data structures reside in device memory, with minor host-device transfers (Doi et al., 23 Jan 2026, Walkup et al., 22 Jan 2026).
• Perturbative extensions: Post-variational PT2 corrections implemented deterministically via hash tables achieve sub-millihartree accuracy efficiently (Tubman et al., 2018).

Resource scaling is dominated by basis size $R$ and excitation connectivity $d$ (matvec), with dense diagonalization scaling as $O(R^3)$ and memory as $O(R^2)$, but iterative methods and data-sparse representations mitigate these for very large $R$. Quantum requirements are minimal—only basis measurement shots are required, not expectation values.

Performance benchmarks document ground-state energies for complex molecular systems in minutes (up to $10^9$ configurations over hundreds of GPUs), with end-to-end scaling closely matched across diverse accelerator architectures (Doi et al., 23 Jan 2026, Walkup et al., 22 Jan 2026).

5. Applications, Limitations, and Generalizations

SBD has been transformative for quantum chemistry (transition-metal clusters, G1 benchmarks), many-body and Bethe Ansatz impurity systems, block decomposition in matrix algebras, and operator-theoretic restricted diagonalization. Key advantages include noise resilience in quantum-classical protocols, systematic improvability by basis growth, variational rigor, and the capacity to address properties beyond energies.

However, limitations persist:

• Basis growth: For quantum many-body regimes lacking ground-state concentration, exponential scaling in required basis size remains, mitigated partially by neural-network-guided adaptive rotations (Cantori et al., 18 Aug 2025).
• Random networks: SBD often provides no dimensionality reduction beyond the trivial, as block structure exists only in the presence of symmetries or repeated modules (Panahi et al., 2021).
• Infinite-dimensional settings: Generalizations to infinite spectra rely on arithmetic-mean-closed ideals for necessary and sufficient criteria, and the nonlinearity of restricted diagonalizable sets limits operator-theoretic closure (Chiumiento et al., 2021).
• Analytical complexity: For integrable models, explicit matrix elements in the selected basis may be analytically intricate (Burovski et al., 2021).

Generalizations encompass sample-based neural diagonalization in adaptive bases (leading to efficiency in delocalized or critical regimes), projection onto truncated Bethe Ansatz or lattice eigenbases for non-integrable spectral studies, and simultaneous block diagonalization for algebraic structure analysis.

6. Summary and Prospects

Selected Basis Diagonalization provides a unifying framework for efficient and variationally rigorous computation of low-lying spectral properties across quantum chemistry, many-body physics, operator theory, and network science. Its versatility derives from flexible basis selection—via sampling, neural modeling, or algebraic commutant computation—and robust error control. Recent advances in algorithmic efficiency (sort-based selection, dynamic masking, GPU matvec pipelines) enable treatment of previously inaccessible system sizes; adaptive-basis methods further extend its reach to highly non-local or entangled regimes.

The method's future development will likely involve improved basis adaptation (via quantum and neural sampling), further integration of hybrid quantum–classical workflows, and expanded application to non-integrable, strongly correlated, and high-dimensional operator-theoretic contexts.