Ghost Penalty Stabilization in Unfitted FEM

• Ghost penalty stabilization is a technique in unfitted finite element methods that adds mesh-dependent terms to manage instabilities caused by small or ill-shaped cut elements.
• It employs derivative jump penalties, gradient projection operators, and mass-scaling to maintain coercivity, good conditioning, and optimal convergence rates.
• Numerical studies confirm that ghost penalties yield uniform control over error and stability, making them effective in diverse applications including fluid–structure interactions and explicit dynamics.

Ghost penalty stabilization refers to a class of stabilization techniques in finite element methods (FEM) for unfitted (or "cut") meshes, designed to restore key numerical properties—such as coercivity, inf-sup stability, and conditioning—that are otherwise compromised by the presence of small or arbitrarily-shaped intersections between the physical domain and the background computational mesh. Ghost penalties operate by adding carefully chosen mesh-dependent terms, typically involving jump penalties of inter-element derivatives and/or norms of appropriately projected differences, on faces or aggregates adjacent to cut regions. Although first developed to address geometric instabilities in immersed methods such as CutFEM and immersogeometric analysis, the concept now encompasses robust mass-scaling for explicit dynamics, divergence-preserving mixed methods, locking-free stabilization via discrete extensions, and even a distinct nonconvex optimization context. A comprehensive synthesis follows.

1. Motivation: Instabilities in Unfitted Discretizations

Unfitted methods (e.g., CutFEM, immersogeometric analysis) embed the physical domain Ω into a fixed background mesh, allowing the physical boundary to intersect cells in an arbitrary manner. Elements cut by the domain boundary—so-called "small cut," "sliver," or "ghost" elements—may have arbitrarily small intersections with Ω. The support of basis functions that lie largely outside Ω then receive vanishingly small mass and stiffness contributions:

• For second-order (elliptic) problems, the largest eigenvalue of the generalized eigenproblem (K,M) associated with the stiffness and mass matrices scales like $λ_{\max}\sim h_\text{cut}^{-2}$, where $h_\text{cut}$ denotes the measure of the cut. For higher-order PDEs, the scaling deteriorates further, rendering explicit schemes impractical as $Δt_\text{crit}\to0$ (Stoter et al., 2023).
• The discrete gradient or higher-order seminorms become uncontrolled in small cut cells, leading to catastrophic ill-conditioning and loss of coercivity, stability, or inf-sup properties (Boiveau et al., 2016, Badia et al., 2021).
• Classical stabilization strategies (penalty, Nitsche’s method) often require penalty parameters proportional to small-cut measures, resulting in artificially stiff discrete systems with persistent cut-size dependence (Stoter et al., 2023).

Ghost penalty stabilization addresses these fundamental pathologies by coupling degrees of freedom across the "ghost" layer surrounding cut cells to regain control of hidden high-frequency modes and enforce global stability properties regardless of cut geometry.

2. Mathematical Formulation and Mechanisms

2.1. Face-Jump or Derivative-Penalty Ghost Operators

A prototypical ghost penalty term for a degree-$p$ polynomial FEM reads

$$g_h(u_h, v_h) = \sum_{F \in \mathcal{F}_G} \gamma\, h_F^{1 - 2p} \int_F [\partial_n^p u_h] [\partial_n^p v_h]\, dS,$$

where $F$ is a face between adjacent cut elements, $[\partial_n^p u_h]$ denotes the $p$-th normal derivative jump across $F$, $\gamma$ is a stabilization parameter ($O(1)$), and $h_F$ is the local face size (Stoter et al., 2023, Boiveau et al., 2016, Burman et al., 2022).

More generally, one may sum over a range of derivatives ($1 \leq l \leq p$), leading to

$$J_h(u_h, v_h) = \gamma_g \sum_{F\in\mathcal{F}_G} \sum_{l=1}^{p} h^{2l-1} \langle [D^l_{n_F} u_h], [D^l_{n_F} v_h] \rangle_F.$$

This symmetric positive semi-definite bilinear form penalizes high-order inter-element discontinuities within a band of cut or ghost cells (Boiveau et al., 2016, Wichrowski, 28 Feb 2025).

2.2. Gradient-Penalty/Projection-Based Ghost Operators

Recent methods introduce ghost penalties based on local projections of the solution gradient:

$$s_{h,k}(u, v) = \int_{\Omega_{h,k}(\delta)} \mu_k [\nabla u_k - \mathcal{P}_k(\nabla u_k)] \cdot \nabla v_k\, dx,$$

where $\mathcal{P}_k$ is a (lumped-) $L^2$ projection into piecewise polynomials and $\Omega_{h,k}(\delta)$ is a cut/collar layer (Olshanskii et al., 28 Jan 2025). This mechanism provides uniform $H^1$-seminorm control independent of cut geometry and is "parameter-free" aside from Nitsche-type penalties.

2.3. Mass-Scaling Ghost Terms in Explicit Dynamics

For time-explicit PDEs (e.g., transient membranes and shells), ghost penalties may be added not only to stiffness operators but also to the mass matrix:

$$M_\gamma(\ddot u_h, v_h) = \gamma_M \sum_{F \in \mathcal{F}_\text{cut}} h_F^{1-2s} \int_F \rho\ [\partial_n^s \ddot u_h] [\partial_n^s v_h]\, dS,$$

with $\gamma_M \sim h^{2p + 1}$ chosen for consistent scaling. Such terms selectively "scale up" the mass of otherwise "light" cut basis functions, mitigating eigenvalue blowup and maintaining convergence rates (Stoter et al., 2023).

2.4. Aggregation and Discrete Extension Ghost Penalties

Ghost penalty operators have been abstracted as functionals penalizing the distance between the solution and its extension from interior (well-posed) regions:

$$s_h^{L^2}(u, v) = \sum_{U \in \mathcal{T}_h^{ag}} \gamma h_U^{-2} (u - \mathcal{P}_h^{ag}u, v - \mathcal{P}_h^{ag}v)_{L^2(U)}.$$

Here, $\mathcal{P}_h^{ag}$ projects onto an aggregated finite element subspace defined by discrete extension from interior DOFs, conferring locking-free robustness in the $\gamma \to \infty$ limit (Badia et al., 2021, Burman et al., 2022).

3. Stability, Convergence, and Locking Behavior

Ghost penalties are constructed to guarantee—uniformly in $h$ and cut geometry—the following critical stability results:

• Extended Coercivity: The stabilized bilinear form dominates the $H^1$ (or higher-order) seminorm over the entire mesh:

$$a_h(u_h, u_h) + s_h(u_h, u_h) \gtrsim \|u_h\|_{V(h)}^2,$$

ensuring the absence of spurious near-kernel modes on small/degenerate cut cells (Boiveau et al., 2016, Frei et al., 21 Mar 2025, Badia et al., 2021).

• Uniform Conditioning: The condition number of the stiffness matrix satisfies $O(h^{-2})$ scaling independent of how the mesh cuts the domain (Stoter et al., 2023, Olshanskii et al., 28 Jan 2025, Wichrowski, 28 Feb 2025).
• Consistency and Convergence: The ghost penalty vanishes for exact solutions (smooth enough), preserving optimal approximation rates ($O(h^p)$ in $H^1$, $O(h^{p+1})$ in $L^2$), provided the penalty acts only on the "bad" regions and is of appropriate scaling (Stoter et al., 2023, Boiveau et al., 2016, Olshanskii et al., 28 Jan 2025).
• Inf-sup Stability in Saddle Point Problems: Stabilization of divergence or pressure jumps extends the inf-sup stability of classical mixed spaces (e.g., Scott–Vogelius, Raviart–Thomas) to unfitted/cut meshes (Liu et al., 2021, Frachon et al., 2022).

A key controversy addressed in recent works is locking: classical ghost penalties, when taken in the infinite penalty limit, enforce global polynomial/algebraic constraints on the solution space, leading to poor approximation (locking) (Badia et al., 2021, Burman et al., 2022). Aggregated FE and so-called "weak AgFEM" approaches design the penalty so that its kernel coincides with a proper locking-free extension space.

4. Ghost Penalty Variants and Methodologies

Variant	Target	Key Formula/Mechanism
Face-jump (derivative)	$H^p$-seminorm, coercivity	Sum of $[\partial_n^p u][\partial_n^p v]$ over faces (Stoter et al., 2023, Boiveau et al., 2016, Wichrowski, 28 Feb 2025)
Gradient-projection	$H^1$-seminorm, algebraic control	$[\nabla u - \mathcal{P} (\nabla u)]$ (Olshanskii et al., 28 Jan 2025)
Mass-scaling	Eigenvalue stabilization (dynamics)	Ghost penalty term in mass matrix (Stoter et al., 2023)
Aggregation/extension	Locking-free, limit behavior	Penalize $u-P^{ag}u$ over aggregates (Badia et al., 2021, Burman et al., 2022)
Divergence/pressure	Mixed methods, inf-sup	Penalize jumps in div $u$ or $p$ (Liu et al., 2021, Frachon et al., 2022)
Optimization-theoretic	Merit functions, analysis	Ghost-penalty "Lyapunov" function, no direct stabilization (Facchinei et al., 2017)

For explicit schemes (membranes/shells), both stiffness and mass matrices may be stabilized by ghost penalties, resulting in cut-size independent critical timestep $Δt_\text{crit} \sim O(h^s)$ instead of $O(h_\text{cut}^{1/2})$ (Stoter et al., 2023). For unfitted Stokes and Darcy flow discretizations, divergence-preserving ghost penalties maintain both conditioning and structural conservation properties (Liu et al., 2021, Frachon et al., 2022). In penalty-free Nitsche formulations, ghost penalties are essential to guarantee full inf-sup stability and avoid suboptimal convergence (Boiveau et al., 2016).

5. Implementation Strategies and Practical Guidelines

Ghost penalty terms are practically implemented as additional bilinear forms, typically assembled only on faces, cells, or aggregates in a small neighborhood of cut or sliver elements:

• Parameter Choice: $\gamma$-parameters are $O(1)$ in practice; consistency scaling requires $h$-dependent exponents linked to polynomial order and target norm (Boiveau et al., 2016, Liu et al., 2021, Frei et al., 21 Mar 2025). Weight functions (e.g., $w(\kappa_T)$) amplify penalty on very small cut elements (Frei et al., 21 Mar 2025).
• Localization: Restrict stabilization to a few layers around the interface (macro-elements or ghost band) to reduce computational cost and preserve global accuracy (Frachon et al., 2022).
• Matrix-Free/High-Order: Tensor-product factorization allows efficient matrix-free application at $O(k^{d+1})$ complexity, critical for high-order unfitted methods (Wichrowski, 28 Feb 2025).
• Parallelization: Careful partitioning ensures that each ghost-face penalty is assembled exactly once, enabling scalable implementations in libraries such as deal.II (Wichrowski, 28 Feb 2025, Frei et al., 21 Mar 2025).

Recommended settings (for typical CutFEM/immersogeometric simulations) are: ghost-penalty parameters $\gamma\sim0.01$–$1$, Nitsche interface penalties $\eta\sim h^{-1}$ (or larger for coercivity), selective mass scaling for higher-order methods or explicit dynamics (Stoter et al., 2023, Olshanskii et al., 28 Jan 2025, Frei et al., 21 Mar 2025).

6. Numerical Evidence and Application Domains

Numerical studies consistently demonstrate:

• Uniform control of error and conditioning as the interface sweep through the mesh, even in presence of near-degenerate cuts and high anisotropy (Stoter et al., 2023, Anselmann et al., 2021, Frei et al., 21 Mar 2025, Boiveau et al., 2016).
• Optimal convergence rates in both $H^1$ and $L^2$ norms, matching those obtained with body-fitted meshes, provided penalties and boundary corrections are chosen appropriately (Olshanskii et al., 28 Jan 2025, Liu et al., 2021).
• For explicit dynamics, order-of-magnitude increases in $Δt_\text{crit}$ (5–15×) when ghost mass is used, with no adverse effect on error or convergence (Stoter et al., 2023).
• Divergence-preserving ghost penalties enable pointwise conservation properties (e.g., $div\,u_h=0$ up to machine precision for $RT_k\times Q_k$) while maintaining well-posedness and optimal approximation rates (Frachon et al., 2022).
• In fluid–structure interaction with contact, ghost penalties prevent spurious solution oscillations, loss of resolution in lubrication layers, and even unphysical interpenetration, stabilizing the monolithic CutFEM scheme across highly challenging benchmarks (Frei et al., 21 Mar 2025).

7. Extensions, Limitations, and Theoretical Analyses

Recent developments include:

• Locking-Free Stabilization: Ghost penalties formulated as penalization with respect to aggregated finite element extensions ("weak AgFEM") eliminate the locking observed in the strong penalty limit for classical face/bulk penalties; the kernel coincides with the locking-free aggregated FE space (Badia et al., 2021, Burman et al., 2022).
• Functional (DOF-Based) Ghost Penalties: Allow arbitrary stabilization functionals, particularly evaluation at degrees of freedom, which delivers more localized and precise control (Burman et al., 2022).
• Extension to Moving Domains and Space-Time: Ghost penalties ensure robust extension of physical variables and stabilization in time-dependent CutFEM for evolving domains (Anselmann et al., 2021).
• Optimization-Theoretic Use: In nonconvex constrained optimization, "ghost-penalty" refers not to direct stabilization but to a merit function used in convergence theory—a conceptual rather than algorithmic device (Facchinei et al., 2017).

Limitations of classical ghost-penalty forms include possible suboptimality in the $L^2$-norm for certain penalty-free Nitsche variants (loss of adjoint consistency) (Boiveau et al., 2016), and strong dependence on the appropriate scaling of the stabilization parameters.

---

The ghost penalty paradigm is now established as a cornerstone of robust unfitted finite element methodologies in computational mechanics, supporting optimal rates, uniform conditioning, and rich model generality in the vicinity of interfaces, severe geometric degeneracies, and for a wide range of PDE systems (Stoter et al., 2023, Boiveau et al., 2016, Badia et al., 2021, Olshanskii et al., 28 Jan 2025, Liu et al., 2021, Frachon et al., 2022, Wichrowski, 28 Feb 2025, Frei et al., 21 Mar 2025, Anselmann et al., 2021).