JacNet: Jacobian Neural Function Approximation

• JacNet is a neural architecture that approximates functions by directly learning their Jacobian fields, enabling robust enforcement of constraints like invertibility and Lipschitz continuity.
• It employs structured Jacobian parameterizations, such as positive-definite and spectral-norm bounded formulations, to guarantee controlled derivative properties and theoretical guarantees.
• The network reconstructs functions through numerical integration of the Jacobian, facilitating accurate forward and inverse computations in supervised learning settings.

JacNet is a neural architecture designed for function approximation by directly parameterizing the Jacobian field of the target mapping, rather than the function itself. The key insight is that by learning a structured Jacobian, constraints such as invertibility and Lipschitz continuity can be imposed and controlled through neural network activations and output parameterizations, enabling the encoding of qualitative functional priors into neural models. Integration of the network-predicted Jacobian reconstructs the original function, and invertibility is ensured—or $k$-Lipschitz priors enforced—by appropriate structural modifications to the network output. This approach provides fine-grained control over the differentiable properties of the learned mapping, facilitating both forward and inverse function computations with theoretical guarantees (Lorraine et al., 2024).

1. Core Principle: Jacobian Field Learning

JacNet targets the supervised learning problem where one seeks to model an unknown $C^1$ diffeomorphism $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$. Rather than directly learning $f_\theta(x)$ as a neural network, JacNet posit that the Jacobian $J_\theta(x) = \partial f_\theta(x) / \partial x$ is instead learned via a separate network. Using the fundamental theorem of line integrals, the function $f_\theta(x)$ can be reconstructed from its Jacobian field as: $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$ where $x_0$ is a fixed base-point with known value.

The learning objective is empirical risk minimization over the data: $$\bar{R}(\theta) = \frac{1}{n} \sum_{i=1}^n L(y_i, f_\theta(x_i)),$$ where $L$ is a loss (e.g., $C^1$0 or $C^1$1), optimized via backpropagation through a numerical integration procedure.

This construction allows typical neural network machinery—activation functions, normalization constraints, positive-definite factorizations—to directly operate on the Jacobian, thereby imparting desired structural priors onto the function's derivatives.

2. Structuring the Jacobian Output

The network parameterizing $C^1$2 can be constructed to enforce particular structural properties essential for qualitative function control:

• Unconstrained Jacobian: $C^1$3.
• Positive-Definite (PD) Jacobian for Invertibility: Let $C^1$4 be an unconstrained output, then

$C^1$5

for small $C^1$6. This guarantees strict positive-definiteness, ensuring all eigenvalues $C^1$7 and $C^1$8, precluding negative eigen-directions.

• Block-Triangular or Triangular Factorization: (Proposed for future work.) The network outputs a lower-triangular matrix $C^1$9 (with positive diagonals via softplus activations) and sets $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$0. This again ensures PD-ness and invertibility.
• Spectral-Norm-Bounded (Lipschitz) Jacobian: To enforce a $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$1-Lipschitz bound,

$$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$2

where $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$3 is unconstrained; the elementwise tanh scales entries to $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$4 and provides $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$5 (tightened to $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$6 with spectral normalization).

These constructions allow invertibility, $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$7-Lipschitz, and more general derivative-based priors to be imposed via architectural choice.

3. Theoretical Guarantees: Invertibility and Properness

JacNet's approach is supported by the inverse function and Hadamard's global inverse theorems. Specifically, if $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$8 is $$f: X \subset \mathbb{R}^d \to Y \subset \mathbb{R}^d$$9 and $f_\theta(x)$0 for all $f_\theta(x)$1, then $f_\theta(x)$2 is locally invertible; if $f_\theta(x)$3 is also proper (i.e., pre-images of compact sets are compact) and the domains are connected, $f_\theta(x)$4 is a global $f_\theta(x)$5-diffeomorphism.

In JacNet, parameterizing $f_\theta(x)$6 as strictly PD via the method above enforces $f_\theta(x)$7. Properness can be ensured by domain clamping or by constructing $f_\theta(x)$8 to be coercive (i.e., $f_\theta(x)$9 as $J_\theta(x) = \partial f_\theta(x) / \partial x$0), which is often achieved in practice.

4. Enforcing Lipschitz and Additional Priors

A function is $J_\theta(x) = \partial f_\theta(x) / \partial x$1-Lipschitz if $J_\theta(x) = \partial f_\theta(x) / \partial x$2 for all $J_\theta(x) = \partial f_\theta(x) / \partial x$3. By the mean-value inequality, a sufficient condition is: $J_\theta(x) = \partial f_\theta(x) / \partial x$4 JacNet realizes this by bounding the spectral norm of the output Jacobian $J_\theta(x) = \partial f_\theta(x) / \partial x$5 through:

• Elementwise tanh scaling, plus spectral normalization on intermediate weights.
• Direct orthonormal factorization with singular values clamped to $J_\theta(x) = \partial f_\theta(x) / \partial x$6.
• Householder or Cayley-based parameterizations.

Because the Jacobian is directly output by the network, these constraints are applied as output layer activations, leaving the reconstruction procedure by line integration unchanged. Other priors, such as convexity or Cauchy–Riemann conditions, can be encoded similarly.

5. Training and Optimization Procedure

Training JacNet involves choosing a base-point $J_\theta(x) = \partial f_\theta(x) / \partial x$7 (and $J_\theta(x) = \partial f_\theta(x) / \partial x$8 if available), defining the reconstructed function $J_\theta(x) = \partial f_\theta(x) / \partial x$9 via: $f_\theta(x)$0 The training objective is: $f_\theta(x)$1 where $f_\theta(x)$2 may be a $f_\theta(x)$3 penalty or additional regularization, but experiments are conducted with $f_\theta(x)$4. Optimization leverages backpropagation through the numerical integrator for end-to-end learning.

6. Inverting the Learned Mapping

Given that $f_\theta(x)$5 is a global diffeomorphism under the enforced constraints, inversion is performed by the analogous integral in output space: $f_\theta(x)$6 where $f_\theta(x)$7. For the matrix-factorized Jacobian form, inversion is efficiently computed via Cholesky or Woodbury identities (complexity $f_\theta(x)$8, reduced to $f_\theta(x)$9 for low-rank $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$0).

An Euler scheme is proposed for numerical inversion: $x_0$6

7. Empirical Illustration and Comparative Performance

JacNet is empirically validated on 1D problems, including invertible and Lipschitz function approximation:

• Invertible task ($$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$1, $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$2): Using the positive-definite parameterization, JacNet recovers $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$3 precisely after 50 Adam iterations on $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$4 training points. The unconstrained baseline fails, collapsing its derivative to zero and making inversion intractable.
• 1-Lipschitz task ($$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$5, $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$6): With $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$7 so $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$8, JacNet matches $$f_\theta(x) = f(x_0) + \int_0^1 J_\theta(x_0 + t(x-x_0)) \cdot (x - x_0)\, dt,$$9 exactly with bounded slope; the unconstrained baseline fails to maintain Lipschitz bounds and generalizes poorly.

Results are summarized as follows:

Task	Unconstrained	JacNet-PD inv.	JacNet-Lipschitz
$x_0$0 inv.	$x_0$1	$x_0$2	N/A
$x_0$3 Lipschitz	$x_0$4	N/A	$x_0$5

Explicit control of derivative properties, as provided by JacNet, is demonstrated to be essential for correct qualitative behavior (monotonicity, slope-bounds), not merely for minimizing training error.

8. Functional Priors and Modeling Implications

JacNet conceptually unifies a wide range of structural constraints as output constraints on the learned Jacobian field. By this construction, entire families of functional priors (invertibility, Lipschitz continuity, convexity, Cauchy–Riemann, among others) can be incorporated directly into end-to-end differentiable models. This allows modeling of highly flexible, universal-approximation-style derivatives, with precise control over the qualitative behavior of the resulting function mappings (Lorraine et al., 2024).