Memory-aware end-to-end learning of channel distortions in optical coherent communications

Vladislav Neskorniuk*, Andrea Carnio, Domenico Marsella, Sergei K. Turitsyn, Jaroslaw E. Prilepsky, Vahid Aref

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


We implement a new variant of the end-to-end learning approach for the performance improvement of an optical coherent-detection communication system. The proposed solution enables learning the joint probabilistic and geometric shaping of symbol sequences by using auxiliary channel model based on the perturbation theory and the refined symbol probabilities training procedure. Due to its structure, the auxiliary channel model based on the first order perturbation theory expansions allows us performing an efficient parallelizable model application, while, simultaneously, producing a remarkably accurate channel approximation. The learnt multi-symbol joint probabilistic and geometric shaping demonstrates a considerable bit-wise mutual information gain of 0.47 bits/2D-symbol over the conventional Maxwell-Boltzmann shaping for a single-channel 64 GBd transmission through the 170 km single-mode fiber link.
Original languageEnglish
Pages (from-to)1-20
JournalOptics Express
Issue number1
Early online date19 Dec 2022
Publication statusPublished - 2 Jan 2023

Bibliographical note

Published by Optica Publishing Group under the terms of the Creative Commons Attribution 4.0 License []. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Funding: H2020 Marie Skłodowska-Curie Actions (766115); Engineering and Physical Sciences Research Council (EP/R035342/1); Leverhulme Trust (RP-2018-063).


Dive into the research topics of 'Memory-aware end-to-end learning of channel distortions in optical coherent communications'. Together they form a unique fingerprint.

Cite this