Computing leaky modes of optical fibers using a FEAST algorithm for polynomial eigenproblems

Abstract: An efficient contour integral technique to approximate a cluster of nonlinear eigenvalues of a polynomial eigenproblem, circumventing certain large inversions from a linearization, is presented. It is applied to the nonlinear eigenproblem that arises from a frequency-dependent perfectly matched layer. This approach is shown to result in an accurate method for computing leaky modes of optical fibers. Extensive computations on an antiresonant fiber with a complex transverse microstructure are reported. This structure is found to present substantial computational difficulties: Even when employing over one million degrees of freedom, the fiber model appears to remain in a preasymptotic regime where computed confinement loss values are likely to be off by orders of magnitude. Other difficulties in computing mode losses, together with practical techniques to overcome them, are detailed.