Nonlinear Signal-Noise Interaction in Optical Fiber Transmission

In a long-haul optical fiber communication system, fiber attenuation, dispersion, and nonlinearity combine with non-deterministic noise from optical amplifiers used for periodic regeneration and cause adverse effects on system performance. In this dissertation, we study and mitigate such undesirable effects.

We present a modified nonlinear decision feedback equalizer designed for use in a legacy optical communication system with periodic dispersion compensation. The effects of noise and nonlinearity on the equalizer coefficients are investigated, and a suboptimal convergence algorithm to reduce such effects is proposed and verified.

Noting the limited ability of existing signal processing tools to combat signal-noise nonlinear interaction effects, we next consider a fundamental scenario to study these effects. We apply Gaussian mixture modeling (GMM) techniques to better understand how noise interacts with the signal in a nonlinear optical fiber span. We validate our technique and learn that at higher levels of nonlinearity, the GMM analysis is more accurate than an additive Gaussian noise or a Volterra series transfer function model.

Finally we present an approach to generalize our analysis. We validate our claims that transmitting a small number of pulses is a good approach to predict the analysis of a practical communication system. We also show that using a 3- or higher-order GMM is necessary to fully understand the nonlinear interaction.