Virtual Extended Formulations

• Virtual extended formulations are an innovation in polyhedral theory that represent a polytope as the Minkowski difference of two polytopes, generalizing classical extension complexity.
• They capture the computation of piecewise-linear neural networks by decomposing support functions into differences of convex functions, bridging LP methods with neural network expressivity.
• This framework enables efficient optimization through dual LP strategies and inspires novel lower-bound techniques in combinatorial optimization and computational complexity.

A virtual extended formulation is a formal innovation in polyhedral theory and optimization, capturing the expressive power needed to represent the linear optimization problem over a polytope not just as a linear program, but as the difference of two linear programs. This perspective generalizes the classical notion of extension complexity and provides a sharper tool for understanding the representational strength of systems such as neural networks with piecewise-linear activations, where exact computation naturally involves differences of convex functions. Virtual extension complexity thereby opens new directions for both lower-bound techniques in computational complexity and for efficient algorithmic strategies in combinatorial optimization.

1. Foundations: Classical and Virtual Extended Formulations

The extension complexity ($\xc(P)$) of a polytope $P \subseteq \mathbb{R}^d$ is the minimum number of linear inequalities (facets) needed in any higher-dimensional polytope $Q$ that linearly projects onto $P$. Equivalently, $\xc(P)$ formalizes the cost of modeling $P$ via linear programming.

Virtual extension complexity ($\vxc(P)$) strengthens this concept by recognizing that many computational objects—especially those arising in neural network expressivity—can represent $P$ not directly, but as a formal Minkowski difference: $P + Q = R$, for polytopes $Q, R$. The virtual extension complexity is then defined as

$\vxc(P) = \min\big\{\,\xc(Q) + \xc(R)\ \big|\ Q, R \ \text{polytopes},\ P + Q = R\,\big\}.$

This allows one to encode optimization over $P$ as the difference of two linear-programming problems—over $R$ and over $Q$—where the computation of the support function $f_P(c) = \max_{x \in P} c^\top x$ is realized as $f_R(c) - f_Q(c)$. Immediately, $\vxc(P) \leq \xc(P)$, with equality whenever $Q = \{0\}$, $R = P$.

2. Virtual Extended Formulations and Neural Networks

The rationale for virtual extended formulations arises from the limitations of ordinary extension complexity in characterizing the expressive power of general neural networks, particularly those with ReLU or maxout activations. Such networks can compute arbitrary differences of convex, continuous piecewise-linear (CPWL) functions. Consequently, lower-bounding the network size solely in terms of $\xc(P)$ risks substantial looseness for non-monotone (general) networks.

The main theorem in this context establishes a direct connection: for every polytope $P$, if $\nnc(P)$ is the minimal size of rank-2 maxout (or ReLU) networks exactly computing the support function $f_P$, then

$\vxc(P) \leq 4\,\nnc(P), \quad\text{hence}\quad \nnc(P) \geq \tfrac{1}{4}\,\vxc(P).$

The proof decomposes any rank-2 network computing $f_P$ into monotone components, each corresponding to a valid LP formulation of the epigraph, whose sizes are then bounded in the virtual EF sum. For strictly monotone networks, ordinary extension complexity suffices with $\xc(P) \leq 2\,\mnnc(P)$, tightly coupling network size and LP representation.

This introduces virtual extension complexity as the fundamental lower bound for general CPWL network size and motivates its study as an independent polyhedral invariant.

3. Connections with Randomized Protocols and Expectation-Based EFs

The development of randomized communication protocols for computing slack matrices in expectation provides a potent framework unifying extension complexity, nonnegative matrix factorizations, and communication complexity. In this model, a protocol is a binary tree where participants, using private randomness and only their own inputs, coordinate messages such that the expected output reconstructs the entries of a nonnegative matrix $M$.

For a polytope's slack matrix $S$, the log of the nonnegative rank, $\log_2 \rank_+(S)$, is exactly the minimum complexity (bits exchanged) in such protocols. This yields a corollary: $\log_2 \xc(P) = \min\, \big\{\ \text{protocol complexity}\ |\ \mathbb{E}[\text{output}] = S\ \big\}.$ This equivalence demonstrates that allowing high-variance (randomized) protocols can realize polynomial-size extended formulations for some polytopes, while deterministic or low-variance protocols drive the necessary number of inequalities exponential. For the perfect matching and spanning tree polytopes, this captures the exponential lower bounds for deterministic models and nuances for randomized ones (Faenza et al., 2011).

Virtual extended formulations, viewed through this protocol lens, can be interpreted as enabling “expectation-based” encodings—melding randomized protocols with difference constructions to minimize size.

4. Examples, Bounds, and the Role of Minkowski Sums

Virtual extension complexity is concretely illustrated by examining decompositions of generalized permutahedra and matroid base polytopes. Every generalized permutahedron $P$ admits a Minkowski sum decomposition $P + Q = \Pi_n$, where $\Pi_n$ (the regular permutahedron) has extension complexity $\Theta(n \log n)$. For explicit matroid base polytopes with $\xc(P) = 2^{\Omega(n)}$, there exist $Q$ such that the extension complexity of the target ($\Pi_n$) is exponentially smaller than that of $P$. However, because virtual extension complexity sums the complexities of both $Q$ and the target $R$, these examples do not yield a separation between $\vxc(P)$ and $\xc(P)$ unless the sum is strictly less than what could be obtained from a single LP. No such strict separation is currently known; exhibiting one remains an open research problem (Hertrich et al., 2024).

For perfect matching polytopes, the known exponential lower bounds on ordinary extension complexity extend (via protocol arguments) to monotone networks, while for general networks only bounds via virtual extension complexity are generic, and their extremality remains unresolved.

5. Algorithmic Consequences and Efficient Optimization

Given a virtual EF of $P$ with polytopes $Q$ and $R$ ($P + Q = R$), one can solve the linear optimization problem over $P$ efficiently by solving two LPs: first over $R$, then over $Q$ (using lexicographic tie-breaking). For a generic objective $c$, the optimizer satisfies $x^R = x^P + x^Q$, so the difference gives an optimal solution for $P$. The total LP size is $\xc(Q) + \xc(R)$, and both can be solved in polynomial time in their sizes.

This shows that small virtual EFs not only encode the combinatorial structure of $P$ but also enable efficient algorithmic schemes for optimization in high-dimensional settings—complementary to, and sometimes more powerful than, direct single-LP formulations (Hertrich et al., 2024).

6. Open Problems and Future Perspectives

Key research directions regarding virtual extended formulations include:

• Can one construct explicit polytopes $P_n$ realizing a provable strict gap $\vxc(P_n) \ll \xc(P_n)$? Such a separation would imply “two LPs minus one LP” are strictly more capable than one LP alone, for some problems.
• What are effective lower-bound techniques for virtual extension complexity? All current lower-bound tools inherently target traditional EFs, not virtual ones.
• For natural polytopes associated with NP-hard problems, is virtual extension complexity always exponential, or could there exist superpolynomial but subexponential instances?
• How does virtual extension complexity inform the representational and computational power of non-monotone neural networks in high-dimensional combinatorial optimization tasks?

A plausible implication is that a systematic study of virtual EFs may reveal fundamental limitations and trade-offs of both polyhedral and neural-computational approaches to optimization, with potential consequences for complexity theory and practical algorithm design. The lack of known separations and effective lower-bound methods underlines the significance and challenge of advancing this field.