Symmetry breaking and instability for semilinear elliptic equations in spherical sectors and cones

Abstract: We consider semilinear elliptic equations with mixed boundary conditions in spherical sectors inside a cone. The aim of the paper is to show that a radial symmetry result of Gidas-Ni-Nirenberg type for positive solutions does not hold in general nonconvex cones. This symmetry breaking result is achieved by studying the Morse index of radial positive solutions and analyzing how it depends on the domain D on the unit sphere which spans the cone. In particular it is proved that the Neumann eigenvalues of the Laplace Beltrami operator on D play a role in computing the Morse index. A similar breaking of symmetry result is obtained for the positive solutions of the critical Neumann problem in the whole unbounded cone. In this case it is proved that the standard bubbles, which are the only radial solutions, become unstable for a class of nonconvex cones.