On symplectic dynamics near a homoclinic orbit to 1-elliptic fixed point

Abstract: We study the orbit behavior of a four dimensional smooth symplectic diffeomorphism $f$ near a homoclinic orbit $\Gamma$ to an 1-elliptic fixed point under some natural genericity assumptions. 1-elliptic fixed point has two real eigenvalues out of unit circle and two others on the unit circle. Thus there is a smooth 2-dimensional center manifold $W^c$ where the restriction of the diffeomorphism has the elliptic fixed point supposed to be generic (no strong resonances and first Birkhoff coefficient is nonzero). Moser's theorem guarantees the existence of a positive measure set of KAM invariant curves. $W^c$ itself is a normally hyperbolic manifold in the whole phase space and due to Fenichel results every point on $W^c$ has 1-dimensional stable and unstable smooth invariant curves forming two smooth foliations. In particular, each KAM invariant curve has stable and unstable smooth 2-dimensional invariant manifolds being Lagrangian. The related stable and unstable manifolds of $W^c$ are 3-dimensional smooth manifolds which are supposed to be transverse along homoclinic orbit $\Gamma$. One of our theorems presents conditions under which each KAM invariant curve on $W^c$ in a sufficiently small neighborhood of $\Gamma$ has four transverse homoclinic orbits. Another result ensures that under some Birkhoff genericity assumption for the restriction of $f$ on $W^c$ saddle periodic orbits in resonance zones also have homoclinic orbits though its transversality or tangency cannot be verified directly.