Flex Ethernet: breaking the chains of physical bandwidth

Technical Features

Ethernet networks have grown from connecting buildings to connecting cities, countries and continents. To move increasingly more information, service providers, hyperscale companies and data centres are deploying new Ethernet links. Some of the challenges they face include:

-Different types of physical interface (transceiver) technologies (CFP4, QSFP28, CFP8) make it difficult for architects and designers to adapt to current and future requirements

-Mature information rates being widely deployed (e.g., 100G) and advanced new rates becoming standardised

-Generation of new technology ecosystems

-Maximising the utilisation of existing infrastructures to fully profit from prior investment before spending CAPEX on new technology

-Combination of transport technologies mixing Ethernet and coherent optical transmission techniques

In the eye of this storm comes FlexE (Flex Ethernet). FlexE breaks the chains imposed by physical interfaces while providing a solution to the challenges listed above.

FlexE defined

FlexE is an implementation agreement developed in 2016 by the Optical Internetworking Forum. It describes a mechanism to transport a range of different Ethernet MAC rates not based on any current Ethernet PHY rate, breaking the constraints of transporting traffic limited to specific interface capacities.

One objective of FlexE is to improve and maximise the interconnection between network elements (routers) and transport gear. A second objective is to help network elements reach the bandwidth currently handled today by transport elements that use coherent technologies.

Bonding, sub-rating and channelization

The main actions that a network element can perform with FlexE are listed below.

Bonding: Creates high capacity data pipes by assigning several Ethernet PHYs to transport MAC rates greater than 100G. (see Figure 1)

It’s a very good alternative to layer 3 aggregation technologies like LAG, and ECMP that are being used to aggregate 100G links, but use former hashing algorithms that can’t completely saturate the link and are difficult to control.

Sub-rating: An action that allows network elements to sub-divide physical interfaces in order to transport lower data pipes over partially filled Ethernet PHYs. (see Figure 2)

Some applications of sub-rating are attempting to mimic transmission technologies such as coherent optical transmission. The purpose is to transport specific rates per wavelength that are not Ethernet standardised.

Channelization: Channelisation provides network elements with the capability to create specific transport channels for multiple data pipes though associated Ethernet PHYs, on the same or even different directions. (see Figure 3)

Potentially this could be a layer 2 substitute for current technologies like LSP on MPLS or even VLANs inside switching networks, in order to create specific paths for Ethernet traffic.

Figure 3. FlexE channelization example

FlexE applications

FlexE may still seem like a very new technology with still-unproven potential. However, the applications where this technology could prove important are growing every day.

Network equipment manufacturers (NEMs): NEMs are working hard to deliver equipment capable of generating different data pipes to transport FlexE clients, and most importantly, accomplish FlexE tasks like bonding, sub-rating and channelisation. Today, NEMs already require test equipment with FlexE features to help them produce the next generation of routers and switches.

Data centers: FlexE’s bonding capability will allow data centres to easily increase bandwidth leaving the datacentre, efficiently aggregating and saturating 100GE PHYs now and 400GE PHYs as they become available. This will replace existing tools like LAG or ECMP.

Subsea cables: Data centres have acquired subsea cables supporting coherent optical transmission technologies. These technologies are able to transport non-standardised Ethernet rates: for example 8QAM modulation, a type of coherent optical transmission that handles 150G per wavelength, a non-standardised rate. Some applications of sub-rating are attempting to match these types of line rates, allowing data centres to take full advantage of their current infrastructure.

Service providers: Inside service provider networks, very different technologies coexist to transport traffic. Some networking schemes interchange VLANs between switches or generate an intricate set of tunnels (MPLS) between network elements to connect several sections of the network. These technologies add headers to the frames and bandwidth control is complex. FlexE channelisation provides a way to manage current and future infrastructure better. Ethernet data pipes (channels) can be easily created, with specific bandwidth characteristics and direction inside the network.

5G networks: Due to its granularity, FlexE is not only providing alternatives for high-speed interconnectivity, it’s being considered as a solution to the connectivity challenges of 5G. FlexE could address 5G bandwidth problems by using bonding and channelisation to efficiently aggregate low rate CPRI/eCPRI Ethernet clients (e.g., 10G, 25G) or potentially aggregate new standards (e.g., 50G) from the backhaul to the 5G core. In such circumstances, traffic coming from each residential user or enterprise line would become a FlexE client once they enter the network. (see Figure 4)

Conclusion

FlexE is opening the door for new client rates; generating new models that adapt more efficiently to the constant changes in technology standards and the transceiver evolution; and generating future-proof designs ready for upcoming standards.

By Anabel Alarcon
Product Specialist, EXFO