Pushing The Boundaries Of Webscale Optical DCI Performance

SPONSORED: For nearly 150 years, we’ve been doing our best to squeeze messages into light. From the photophone in the 1880s, which used light to relay voice communications, to the first fiber optic cables in the 1980s, light has been a fast and reliable data communications medium. The better we get at sending messages through underground and undersea glass, the more data people want to use it. But the challenge has always been how to increase the capacity to match data growth.

The volume of information being delivered, stored, and processed in the cloud continues to expand, leaving datacenter operators needing to find new ways of transferring masses of digital data in and out of their facilities as fast as possible to avoid application and service performance bottlenecks. Datacenters now need to exchange many terabits of data every second, and keeping up with that demand takes skill, sophistication, and silicon. This places optical datacenter interconnect (DCI) solutions that sends traffic directly from one hosting facility or datacenter to another as perhaps the single most critical part of these global data networks.

Optical DCI requirements break down into three main segments, explains Serge Melle, who leads optical networks product marketing at Nokia. The first is connecting large datacenters or internet exchanges within a metropolitan area, effectively turning them into one larger virtual facility. "Then there's also a need to interconnect datacenters across countries and even across continents." he says.

For years these two were the primary use cases, but then things changed as edge computing evolved and cloud providers moved their services closer to the edges of the network.

"Datacenter operators are using infrastructure from other providers, often called neutral or digital infrastructure providers, to handle this challenge," says Melle. "Reducing backhaul distances and processing data closer to the edge increases service quality, but you still need to move data to those edge locations from the remote datacenters in order to support local metro peering."

The companies building those connections face two main challenges, the first and most important of which is capacity. Whether it's interconnecting data centres in the metro, across long-haul distances, or bringing them closer to the edge – all of these use cases need very high scalability.

The second is less obvious: energy efficiency. The lion's share of power consumption in datacenters comes from the computers that run in them and the cooling equipment that keeps them within acceptable temperature ranges. But networking power consumption is also important.

"Datacenter operators have a holistic approach covering every aspect of the infrastructure," explains Melle. "They will consider everything that can drive down the power budget, including the power per bit needed to send information between facilities."

When building your own dark fiber-based optical DCI, it's also important to consider the power budget in the connection between the datacenters. Light has traditionally only travelled a maximum of a few hundred kilometres down an optical fiber. So long haul DCI may require repeaters that convert light signals to electricity and back again before sending them back on their way.

Reducing the number of repeaters needed cuts the power budget for those long-distance connections and also reduces the points of failure along the connection. The ideal situation, says Melle, would be not to have any repeaters at all.

The communications industry has taken big steps to increase fiber-based bandwidth. It has already moved from direct detect, which uses transmission power to communicate bits by adjusting the amplitude of the light, to coherent optics, which adjusts the phase of the light signal.

But there's still plenty of room to improve communications capacity up to the Shannon Limit. Named after its inventor, information theorist Claude Shannon, this limit represents the maximum rate at which a signal can travel along a medium given two factors: bandwidth (the range of frequencies available on that channel) and noise (interference).

Nokia's effort to push DCI rates close to the Shannon limit lies in the optical module that converts electrical signals to light and sends them down the cable. The company has been evolving the technologies to enable this, including the digital signal processor that does enables the electrical to light conversion, using its Photonic Service Engine (PSE) chipset.

"Pushing the limit of the DSP's speed gives you these performance improvements," Melle says. Aside from driving up chip clock speeds through smaller fabrication die sizes, the company also includes some other elements in its chip design. One of these is called Probabilistic Constellation Shaping (PCS), developed by Nokia's research and development company, Nokia Bell Labs.

"PCS continually adapts the modulation of the coherent signal to either maximise the number of bits transmitted or minimise the amount of spectrum being used," explains Melle. "This goes back to the Shannon limit, which defines the maximum bits per spectrum you can transmit. With probabilistic constellation shaping, you can push that close to the Shannon limit along any given distance."

Nokia's other focal point is integration, for which it must bring the DSP that performs the electrical-to-optical conversation closer to the optics used to transmit the light signal. It's no longer acceptable to connect DSPs and optics on a printed circuit board via a copper trace, because it generates too much electrical noise. "Instead, you have to integrate them into a photonic multi-chip module that stacks the DSP chip and the silicon photonics together." adds Melle.

These solutions are solving real-world problems for customers. Northern European communications provider GlobalConnect used Nokia's PSE technology to connect its datacenters over thousands of kilometres of fiber optic cable. It needed to scale as the market grew for high-capacity, low-latency connections. It chose Nokia's fifth-generation PSE-V to create a single large network connecting its domestic optical infrastructures in different regions across Europe.

The PSE-V increased the communication speed between GlobalConnect's long-haul points of presence from 10Gbit/sec to 400Gbit/sec, significantly increasing its network capacity. Since it upgraded the network, the company has now increased the speed of its Nordics-based DCI to 600Gbit/sec.

February 2023 saw Nokia launch the sixth generation of its PSE chipset, which it says increases optical performance threefold in terms of the distance over which data can be transmitted without repeaters. And Melle reckons the chipset also reduces power consumption by over 60 percent compared to the prior model.

The new PSE-6 super-coherent optics operates at speeds up to 1.2Tb/s per wavelength, and two wavelengths can be operated as a single channel with a maximum transfer rate of 2.4Gbit/sec to transport up to three 800 Gigabit Ethernet (800GE) services using a single line-card over metro distances. The system can also transmit 800GE services over a single wavelength over long-haul distances.

Nokia recently completed a field trial over GlobalConnect's network demonstrating PSE-6s transmission at 800Gb/s over 2,019km over a single wavelength.. That's critical, Melle says, because before this transmission at 800Gb/s per wavelength was limited to several hundreds of kilometres. Today, network operators are moving to 800GE interfaces in their IP routing networks, putting pressure on optical modules to efficiently support these new Ethernet interface speeds.

Nokia had already developed silicon to support these new IP router interface speeds. In 2021 it introduced its FP5 network processing silicon, which routes network provider traffic with 800GE interfaces to provide end-to-end high-speed throughput.

The PSE also includes some security measures designed to protect client data. One of these is AES 256 encryption, using what Nokia says are ‘quantum-proof’ symmetric keys, hopefully protecting customers from the "quantopocalypse", the point at which computers manipulating qbits will help crack asymmetric encryption. Another feature guards against physical attack by monitoring connections for variations in optical power levels. This could detect cuts made in a fiber cable to eavesdrop on communications.

As appetites for more capacity between datacenters increase, demand for coherent optics having greater scale and better performance show no sign of slowing down. Market analyst Cignal.ai has forecast a growth rate for this technology of over 40 percent per year for the next four years. The ability to squeeze every last drop of communication capacity out of fiber connections – whether metro, edge-based, or via long-haul submarine cables – will be critical for a long time to come.