Branch-and-Bound (BnB) Optimization

• Branch-and-Bound (BnB) is a systematic algorithm for global optimization that recursively partitions the search space and prunes suboptimal regions.
• It employs both classical linear bounds and advanced quasi-lower bounds derived from local quadratic behavior to accelerate convergence.
• Its applications span integer programming, combinatorial optimization, and robust estimation, with confirmed scalability in rigid registration and biological matching problems.

Branch-and-Bound (BnB) is a fundamental algorithmic paradigm for exact global optimization of discrete, continuous, and mixed-integer problems, characterized by recursive space partitioning (branching) and systematic elimination of infeasible/suboptimal subproblems (pruning) via bounding. The method underpins modern solvers in integer programming, combinatorial optimization, robust estimation, and related areas.

1. Core Principles and Classical Framework

The classical BnB framework operates on the following principles:

• Branching: The feasible search space (e.g., a D-dimensional cube in ℝ^D for continuous problems or the polyhedral feasible region for MILPs) is recursively partitioned into smaller subregions. These can be axis-aligned cubes, variable domains, or more general sets, depending on the problem's structure.
• Pruning: At each node (subregion), a fast-to-compute lower bound (or upper bound for maximization) on the objective is computed. If this bound is worse than the current best solution (the incumbent), the branch is discarded. Otherwise, it is recursively subdivided.
• Bounding: For a given region C ⊂ ℝ^D (diameter δ), the classical approach computes a lower bound based on the center x₀:

$$F(x) \geq F(x_0) - \Delta(\delta), \quad \forall x \in C,$$

with the estimate $\Delta(\delta) = L \delta + O(\delta^2)$ commonly derived from Lipschitz or first-order Taylor bounds.

• Complexity: To achieve ε-accuracy (i.e., ensure the global maximum/minimum is bracketed within ε), the bounding granularity must satisfy $\Delta(\sqrt{D}h) \leq \epsilon$ where h is half-edge length. The number of subregions (cubes) required scales polynomially as $N(\epsilon) \sim \epsilon^{-D}$ (with variations between $\epsilon^{-D/2}$ and $\epsilon^{-D}$ depending on the bounding strength) (Dym et al., 2019).

The workflow is summarized as follows:

Step	Operation	Key Input/Output
Branch	Subdivide feasible region	Generate child nodes/regions
Bound	Evaluate lower bound	Discard/prune subregions if bound
Update	Track best upper/lower bounds (incumbent)	Return global optimum within ε

2. Quasi-Lower Bounds and Quasi-BnB

The Quasi-BnB approach introduced by Cohen et al. (Dym et al., 2019) replaces classical linear lower bounds with quasi-lower bounds. These bounds exploit smoothness of objective functions near global minimizers and are based on the local quadratic behavior of the energy landscape.

• Quasi-Lower Bound Definition:

$F(x) \geq F(x_0) - \Delta^*(\delta)$

This bound is guaranteed only under the assumption that the region C contains a global minimizer $x^*$; otherwise, it may overestimate.

• Quadratic Bounds: For locally smooth energies $F(x)$ (arising from minimizing a smooth energy $E(x,y)$ over $y$), Taylor expansion gives near a minimizer:

$$F(x) - F(x^*) \leq C \|x - x^*\|^2 \implies \Delta^*(\delta) = C \delta^2 + O(\delta^3)$$

For a cube of half-edge h and $D$ dimensions: $\delta = \sqrt{D}h$, so

$lb_q(C) = F(\text{center}(C)) - C (\sqrt{D} h)^2$

• Global Safety: Even though quasi-lower bounds are not rigorous lower bounds everywhere, the algorithm remains globally optimal. Non-optimal regions without a global minimizer are eventually pruned by a proper linear bound or subsumed, while the regions containing global minimizers always satisfy the quadratic bound and are retained.

3. Convergence and Complexity Acceleration

The crucial consequence of using quasi-lower bounds is that the convergence rate of BnB improves dramatically.

• Depth Analysis: At refinement generation g, cube diameter $\delta(g) = \sqrt{D}h_0 2^{-g}$
• Termination: $$M 2^{-2g} \leq \epsilon \implies g \gtrsim (1/2) \log_2(M/\epsilon)$$, so depth is $O(\log(1/\epsilon))$
• Total Work: Only a constant number of cubes around each minimizer survive per generation. All others are pruned. Thus, total function evaluations is $O(\log(1/\epsilon))$.

This contrasts with classical BnB, where work is polynomial in $1/\epsilon$ due to the linear bound:

• Classical: $h(g) \sim \epsilon$, number of cubes per depth $2^{Dg} \sim \epsilon^{-D}$

The table below summarizes the differences:

Bound Type	Bounding Term	Cube Refinement	Function Evals
Classical	$L\delta$	polynomial	$\epsilon^{-D}$
Quasi (qBnB)	$C\delta^2$	logarithmic	$O(\log(1/\epsilon))$

4. Quasi-BnB Algorithmic Framework

The Quasi-BnB algorithm (Dym et al., 2019) proceeds as:

1. Initialize a queue with the root cube covering all feasible x.
2. Track best global upper bound (ub) and lower bound (lb).
3. Repeat until $ub - lb \leq \epsilon$:
   • For each cube:
     • Evaluate F at its center $x_C$ ($ub_C$).
     • Compute quasi-lower bound $lb_C = F(x_C) - \Delta^*(\sqrt{D} h_C)$.
     • Update $ub \leftarrow \min(ub, ub_C)$, $lb \leftarrow \min(lb, lb_C)$.
   • Prune cubes with $lb_C > ub$.
   • Subdivide surviving cubes.
4. Return center $x$ giving $ub$.

Key distinctions:

• Bound computation uses quadratic (second-order) terms.
• Pruning is conservative but always safe for global optima.
• Only a logarithmic number of generations of cube refinements are necessary, and almost all cubes are eliminated rapidly except a constant set near $x^*$.

5. Empirical Validation and Applications

Benchmarks confirm significant acceleration and scalability, especially for high-accuracy global optimization tasks:

• Rigid-Closest-Point Registration: On synthetic point clouds and geometric data, quasi-BnB (qBnB) stabilized at approximately $10^6$ function evaluations even as $\epsilon \rightarrow 10^{-6}$, while standard Go-ICP's cost grew as $\epsilon^{-D}$.
• Rigid-Bijective Problems: For 2D and 3D geometric matching, qBnB achieved identical optimal values much faster: e.g., 6.5 seconds vs. failure at 1 hr for classical BnB in complex 3D cases.
• Biological Morphology: For rigid-bijective molar matching, qBnB found globally optimal, biologically plausible solutions in 2–5 min (vs. hours for BnB or suboptimal alignments by local heuristics).

6. Generalization and Theoretical Implications

The quasi-lower bound paradigm is applicable whenever the global objective $F(x) = \min_y E(x, y)$ admits a second-order Taylor expansion around global minima and the Hessian is positive definite. Major domains include:

• Global registration and pose estimation (e.g., for camera and object alignment)
• Mixed-integer nonlinear programming (MINLP) with smooth subproblems
• Any conditionally tractable optimization permitting exact minimization over some variables given others

By replacing Lipschitz (first-order) lower bounds with locally quadratic bounds, one accelerates convergence from polynomial to logarithmic in $1/\epsilon$ (Dym et al., 2019). The only caveat is that the global structure must admit locally clean minima; otherwise, false pruning is possible but does not affect correctness, as any region lacking a minimizer will ultimately be pruned based on the incumbent.

7. Extensions: Shaping, Trimming, and BnB Tree Control

Supplementary BnB strategies further improve practical performance and tree size control:

• Offshoots, Shaping, Trimming: The offshoot-based BnB framework "shapes" the tree by deferring branching order decisions and employs trimming to remove redundant branch variables after infeasibility is detected (Christophel et al., 2017). This approach empirically reduces node counts by 30–50% in MILPs.
• Learning-based Branching: RL and imitation learning methods adaptively select branching decisions (variables or actions), substantially reducing BnB tree size for combinatorial and mixed-integer problems.
• Combined Second-Order and First-Order Bounds: In situations where the function is only locally smooth, the hybrid use of quasi-lower bounds around likely minimizers, reverting to classical bounds elsewhere, balances safety and convergence speed.

These innovations further improve the scalability and adaptability of BnB algorithms to high-dimensional and complex problem classes.

---

Primary Reference: "Linearly Converging Quasi Branch and Bound Algorithms for Global Rigid Registration" (Dym et al., 2019). Supplementary methodology from "Shaping and Trimming Branch-and-bound Trees" (Christophel et al., 2017).