Fiber-Coupled Coherent Communication Systems

• Fiber-coupled coherent communication systems are optical networks that integrate both spatial and temporal information via coherent detection, achieving high data rates and spectral efficiency.
• They employ all-fiber photonic lanterns with coherent beam combining and adaptive phase correction, coupled with advanced DSP techniques for effective impairment mitigation.
• These systems enhance signal-to-noise ratios and turbulence compensation, offering scalable integration for both fiber and free-space optical links.

A fiber-coupled coherent communication system integrates both spatial and temporal optical field information using coherent detection at the fiber interface, enabling high data-rate, spectrally-efficient, and impairment-tolerant optical links. Such systems encompass advanced receiver architectures—most notably all-fiber photonic lantern receivers with in-fiber coherent beam combining (CBC)—as well as sophisticated digital and analog signal-processing chains for impairment compensation, phase recovery, and multiplexing. Recent developments include the integration of adaptive optics, joint spatial-mode/digital processing, and low-complexity DSP for nonlinearity mitigation, establishing all-fiber CBC systems as a core technology for robust, high-performance fiber and free-space optical links.

1. System Architectures and Fiber Coupling

The canonical fiber-coupled coherent receiver for multimode or free-space connections utilizes an all-fiber photonic lantern receiver with CBC (Zhang et al., 2021). The architecture proceeds as follows:

• Optical Input: An FSOC beam, modulated (e.g., BPSK/QPSK), is focused by a telescope and passed through a phase screen simulating turbulence.
• Photonic Lantern: The multimode input (core ≈18 μm, NA ≈ 0.15) is adiabatically split into N single-mode fiber (SMF) outputs.
• Phase-Control Loops: Each adjacent SMF pair feeds a 3 dB coupler; one arm contains a fiber-phase shifter (PS_i) controlled via a PI loop that nulls the destructive port current at a monitoring photodiode (PD_i), implementing adaptive fiber-domain phase correction.
• CBC: After N–1 2×2 SMF couplers in cascade, all N modes are optically phased to interfere constructively into the final SMF, which is then mixed in a SMF coupler with a local oscillator for balanced detection.

The multimode-to-single-mode coupling efficiency is given by

$$\eta_{\mathrm{coupling}} = \frac{\bigl|\,\iint E_{\rm in}(r,\theta)\,\psi^*(r,\theta)\,r\,dr\,d\theta\bigl|^2} {\left[\iint|E_{\rm in}|^2\,r\,dr\,d\theta\right] \left[\iint|\psi|^2\,r\,dr\,d\theta\right]}$$

The aggregate throughput is

$$\eta_{\mathrm{total}} = \eta_{\mathrm{coupling}} \cdot \eta_{\mathrm{lantern}} \cdot \eta_{\mathrm{CBC}}$$

where η_lantern is the MM→SM insertion loss and η_CBC the in-fiber CBC efficiency.

This all-fiber approach requires minimal modification to standard fiber plant, supporting direct integration with SMF links.

2. Coherent Beam Combining and Adaptive Optics

Field summation at the output of the fiber combiner is described algebraically by

$E_{\rm out} = \sum_{i=1}^N a_i e^{j\phi_i}$

where $a_i = \sqrt{P_{S,i}}$ represents the amplitude from the i-th port and $\phi_i$ its controlled phase.

Each phase loop employs a PI controller (gain constants $K_{p,i}$, $K_{i,i}$): $$u_i(t) = K_{p,i} e_i(t) + K_{i,i} \int_0^t e_i(\tau)\,d\tau$$ with loop transfer function $H_i(s)$ and closed-loop sensitivity $T_i(s)$ controlling phase error dynamics.

The SNR advantage of CBC over digital combining is captured by: $$SNR_{\mathrm{CBC}} = \frac{\big|\sum_i a_i\big|^2}{\sigma^2} \geq SNR_{\mathrm{DCC}} = \frac{\big(\sum_i|a_i|\big)^2}{N\sigma^2}$$ Optical CBC delivers up to 3–6 dB SNR improvement when port amplitudes are unequal.

Low-order AO is realized within the fiber domain: phase shifters correct intermodal phase differences induced by turbulence, approximating the correction of the lowest order Zernike phase aberrations by holding the residual phase

$$\Delta\phi_i \simeq \phi_{\text{turb}}(\text{mode}\,i) - \phi_{\text{correction},i}$$

near zero.

3. Performance Characterization and Role of Digital Signal Processing

In proof-of-concept implementations (N=3), the measured lantern coupling efficiency without turbulence was ~0.4, open-loop combiner ~0.23, closed-loop CBC~0.62 (with phase-error variance σ_ϕ²≲0.02 rad²) (Zhang et al., 2021). Under turbulent conditions (D/r₀ = 1.7), closed-loop CBC achieves η_CBC ~0.64. The bit-error-rate for BPSK is

$$BER \simeq \frac{1}{2}\operatorname{erfc}(\sqrt{SNR})$$

with 2–4 dB CBC-induced SNR gain translating to ~10× BER reduction at moderate SNR.

When compared to single-mode fiber direct coupling or photonic-lantern digital coherent combining, optical CBC consistently achieves superior SNR and allows single-photodiode detection while simplifying electronics and fiber integration.

DSP techniques such as weighted FIR filters for time-domain chromatic dispersion equalization, digital carrier-phase recovery (LMS, BWA, VV methods), and DBP are essential for maximizing system reach and spectral efficiency (Zeng et al., 2017, Xu et al., 2016, Civelli et al., 21 May 2025).

4. Nonlinearity Mitigation and Machine Learning Approaches

Kerr and other nonlinearities remain limiting factors in coherent fiber links. DBP (“split-step Fourier method” for the inverse NLSE) can numerically reverse deterministic nonlinear impairments, but at significant computational cost (Civelli et al., 21 May 2025, Choyon et al., 2021). Modern approaches utilize:

• Low-Complexity DBP: ESSFM, coupled-band ESSFM, and learned/FIR-filter–based DBP substantially reduce implementation cost while retaining most of the nonlinear mitigation benefit (Civelli et al., 21 May 2025).
• ML-Based Equalizers: Transformer-based neural architectures are capable of learning inverse nonlinear fiber mappings with complexity per symbol sufficient for real-time ASIC/FPGA integration, offering >1 dB Q-factor improvement and ~20% reach/capacity gains versus DBP in DP-16QAM systems (Gautam et al., 2023).
• Affinity Propagation Soft Clustering: Non-parametric, training-data-free unsupervised methods (AP clustering) have demonstrated up to 5 dB Q-factor improvement and 4 dB power margin extension over standard K-means, Volterra, or DBP NLC in WDM-OFDM systems, at manageable DSP complexity (Giacoumidis et al., 2018).

These methods can be cascaded with linear or CBC front-ends to further improve effective noise tolerance and system capacity.

5. Extensions: Integration, Scalability, and Applications

All-fiber CBC receivers are directly applicable in FSOC links (e.g., satellite–ground, air-to-ground) and can be extended to fiber network nodes for multimode/few-mode modal noise suppression or turbulence-hardened WDM links. Possible extensions include:

• Higher-Port Lanterns/N: Scaling N>5–7 permits capture and correction of higher-order turbulent or modal impairments but increases optical loss and complexity of the CBC network (Zhang et al., 2021).
• Integrated Photonics: Migration from discrete fiber-based phase shifters to thermally or MEMS-controlled photonic circuits (Si/SiN/InP) further reduces device size and complexity, enabling on-chip all-fiber CBC (Geravand et al., 24 Sep 2025).
• DSP-Aided and Hybrid Combining: DSP can estimate residual phase error (with pilot-aided or blind methods), enabling digital feedforward or hybrid optical/electrical combining for maximum flexibility.
• Wavelength and Spatial Multiplexing: WDM can be overlaid on each SMF output of the lantern, and CBC performed independently or jointly per WDM channel or spatial mode, supporting turbulence-tolerant superchannel architectures. Integration of EDWA (erbium-doped waveguide amplifiers) on SiN photonic platforms now supports net 25.6 Tb/s coherent transmission over 81 km in a mm²-scale booster configuration (Che et al., 2024).

Hybrid systems uniting frequency comb sources, spatial division multiplexing, and joint phase recovery (master–slave, multi-channel) can further reduce DSP power and hardware footprint (Lundberg et al., 2019, Geng et al., 2021).

6. Practical Challenges, Trade-offs, and Future Directions

Key implementation challenges are:

• Scalability: Increasing N in the lantern improves turbulence tolerance but demands more phase-control loops and adds insertion loss. Phase actuator bandwidths (currently tens of kHz) can limit AO effectiveness under fast turbulence, motivating MEMS or integrated photonics solutions (Zhang et al., 2021).
• DSP Complexity: Full DBP and high-performance NLC present power and latency bottlenecks; low-complexity DBP and ML-equalizers, especially with transfer learning or windowed operation, provide practical NLC with modest overhead (Civelli et al., 21 May 2025, Gautam et al., 2023, Freire et al., 2021, Ming et al., 2021).
• Insertion Loss Budget: Accumulated loss in lantern, couplers, and integration interfaces sets a lower bound on optical SNR.
• Application Space: All-fiber CBC receivers are most attractive for FSOC, modal-diverse fiber links, and turbulence-compensated switching/routing nodes.

Research is moving toward terabit-scale, highly parallel, and software-defined photonic interconnects with direct integration of all-fiber CBC, on-chip amplifiers, and jointly optimized DSP front-ends leveraging ML, efficient NLC, and joint spatial-wavelength phase estimation.

Advances in hybrid and monolithic Si/SiN photonic platforms now permit ultra-dense shoreline packing and energy-per-bit approaching 10 fJ/bit for O-band coherent transmitters (Geravand et al., 24 Sep 2025), with prospects for >10 Tb/s per fiber in next-generation systems.

7. Summary Table: All-Fiber Lantern CBC vs. Conventional Approaches

Attribute	Lantern + CBC	Lantern + DCC	SMF Direct
Effective SNR (turbulence)	Highest (gains 2–4 dB)	Moderate	Lowest
Phase correction	Optical (in-fiber AO)	DSP only	N/A
Implementation complexity	Moderate; scales with N	Lower	Lowest
Scalability (spatial modes/WDM)	High	Medium	NA
Real-time adaptation	Optical + DSP	DSP only	N/A
Integration prospects	Fiber/integrated photonics	DSP, few-mode	NA

CBC stands out in terms of turbulence/multimode robustness, SNR, and high-density integration feasibility, while maintaining a manageable tradeoff in complexity and insertion loss.

---

For further algorithmic and architectural details on all-fiber CBC receivers and their role within fiber-coupled coherent systems see (Zhang et al., 2021). For associated chromatic-dispersion compensation, nonlinearity mitigation, and DSP integration consult (Zeng et al., 2017, Civelli et al., 21 May 2025, Gautam et al., 2023, Freire et al., 2021, Ming et al., 2021), and for system integration and scaling with amplifiers and frequency comb techniques see (Che et al., 2024, Geravand et al., 24 Sep 2025, Lundberg et al., 2019, Geng et al., 2021).