Roughness-Limited Performance in Ultra-Low-Loss Lithium Niobate Cavities

Abstract: Achieving low optical loss is critical for scaling complex photonic systems. Thin-film lithium niobate (TFLN) offers strong electro-optic and nonlinear properties in a compact platform, making it ideal for quantum and nonlinear optics. While $Q$ factors above $10^7$ have been achieved, they remain below the intrinsic material limit. We present a systematic study of scattering losses due to roughness in TFLN racetrack cavities, isolating contributions from sidewall and interface roughness. Quality factors up to $27 \times 10^6$ are demonstrated in waveguides with widths of $2.2\lambda$ ($\sim3.5\,\mu$m), where interface roughness dominates, and up to $1.2 \times 10^7$ in narrower waveguides $0.8\lambda$ wide ($\sim1.2\,\mu$m), where sidewall roughness is the primary limitation. Our modeling framework, based on 3D wave simulations informed by AFM-measured roughness, is material-independent and broadly applicable across integrated photonic platforms.