The Brezis-Nirenberg problem for the curl-curl operator

Abstract: We look for solutions $E:\Omega\to\mathbb{R}^3$ of the problem $$ \left{ \begin{aligned} &\nabla\times(\nabla\times E) +\lambda E = |E|^{p-2}E &&\quad \text{in }\Omega &\nu\times E = 0 &&\quad \text{on }\partial\Omega \end{aligned} \right. $$ on a bounded Lipschitz domain $\Omega\subset\mathbb{R}^3$, where $\nabla\times$ denotes the curl operator in $\mathbb{R}^3$. The equation describes the propagation of the time-harmonic electric field $\Re{E(x)e^{i\omega t}}$ in a nonlinear isotropic material $\Omega$ with $\lambda=-\mu \varepsilon \omega^2\leq 0$, where $\mu$ and $\varepsilon$ stand for the permeability and the linear part of the permittivity of the material. The nonlinear term $|E|^{p-2}E$ with $p>2$ is responsible for the nonlinear polarisation of $\Omega$ and the boundary conditions are those for $\Omega$ surrounded by a perfect conductor. The problem has a variational structure and we deal with the critical values $p$, for instance, in convex domains $\Omega$ or in domains with $\mathcal{C}^{1,1}$ boundary $p=6=2^*$ is the Sobolev critical exponent and we get the quintic nonlinearity in the equation. We show that there exist a cylindrically symmetric ground state solution and a finite number of cylindrically symmetric bound states depending on $\lambda\leq 0$. We develop a new critical point theory which allows to solve the problem, and which enables us to treat more general anisotropic media as well as other variational problems.