Why the Internet Holds Firm: Internet Infrastructure in Times of Covid-19

The role of the network of subnetworks, once established for research purposes and now known as the Internet, has evolved within a few decades into an omnipresent communication and commercial ecosystem. At the end of 2018, more than 50% of the world’s population, 3.9 billion people from all countries of the world, were using the Internet – and the trend is growing. According to a Cisco[1] forecast, by 2023 there will be 29.3 billion devices worldwide connected to the Internet (or 3.6 devices for every person on the planet), which will send and receive a total of 1,209 Terabits per second (Tbps) of Internet traffic at peak times – the equivalent of about 48 million parallel Netflix 4K streams. According to these estimates, the data traffic of the future will therefore assume enormous proportions.

Covid-19 is changing the use of the Internet – is the Internet reaching its limits?

The major role the Internet now plays in our society is something we are becoming increasingly aware of – especially these days, in the wake of the Covid-19 crisis. Overnight, people were sent to work from home. We started using the Internet to stay in touch with friends and colleagues via video telephony. Small clubs that had previously only had a website began to use streaming to broadcast training sessions, and apart from that, the Internet is simply a source of entertainment in the form of online computer games or video streaming. 

As a result of these changes, a significant increase in data traffic has been recorded at various observation points on the Internet in recent weeks. At the world’s largest Internet Exchange in Frankfurt am Main, Germany, the Internet traffic transmitted at peak times has grown by more than 10% to 9.2 Terabits per second, at smaller Internet Exchanges (measured according to data throughput), such as DE-CIX Dusseldorf, traffic has even grown by more than 20%. In particular, the data traffic for services that are needed for working from home, such as Skype or Zoom, has increased in some cases by up to 100%. Online and cloud gaming traffic has increased by 50%.

In view of these significant changes in our Internet usage behavior, the following questions arise: How much short-term growth can the Internet actually sustain, and what are the limiting factors? The open Internet architecture was, over 50 years ago, initially conceived with important design decisions, such as the technological independence of the individual subnetworks, best-effort packet switching, and no global control. In detail, of course, the Internet and all the technologies involved have undergone enormous development – but many of the basic protocols and concepts (such as IP, BGP and TCP) have mostly only been extended in important details. The totality of these decisions is the basis for the phenomenal growth of the Internet over the last few decades. Therefore, it can easily handle short-term increases in data traffic, as we are currently seeing.

Subnetworks ensure stability

In a simplified representation, the structure of the Internet consists of three different types of subnetworks, each of which represents administrative domains and can thus be directly assigned to individual companies: End-user networks (DSL, UMTS/LTE or cable providers), transport networks, and service provider networks (from where services are transmitted, often known as content delivery networks, or CDNs). In order to be able to consume a video stream, a request from the user is sent to a server of the provider. The data packet is first transported in the end-user network (at the user’s end) to a transfer point, where it is either transferred to a transport network or directly to the network providing the service. These transfer points include Internet Exchanges such as DE-CIX in Frankfurt. If a transport network is connected in between, this ensures delivery through its global backbone.

In order to understand where potential bottlenecks may arise in the current crisis, it is necessary to look more closely at the three components mentioned above: The end-user networks, i.e. those subnetworks that provide us with a broadband connection, such as DSL, UMTS/LTE or cable, transport networks, which in simple terms represent all networks between the end-customer network and the network providing the service, and service provider networks. In a crisis such as the current one, bottlenecks can arise in these three types of networks due to a sharp increase in data traffic.

The limiting factor in the end-user network may, for example, be the connection capacity of the DSL connection, in this case, therefore, more a commercial restriction than a technical one. More important for this question is that end-user network operators must maintain sufficient capacity within their network. Sufficient capacity must be available at all times to transport the necessary data traffic from households or offices through the end-user networks to the transfer points, and from there to other networks (such as transport or service provider networks).

In order to prevent limited data traffic at the end user, not all content in today’s modern Internet is transported directly from the respective CDN servers to the end customer. Frequently, popular content is already made available on servers that are located directly in the end-user network and only need to be transferred there once. A popular new film offered on streaming platforms only has to be transferred once to what is known as a cache server in the end-user network – this applies to up to 50% of cases today. For the final delivery to the customers, the network capacity must still be kept available at the end customer’s end, but this offers considerable savings potential at the network gateways (previously described as transfer points).

Network gateways as bottlenecks 

Potential bottlenecks can also occur at the network gateways. This refers to the critical links and transfer points between the individual subnetworks (end-user networks, transport networks, and service provider networks) which unite the entire ecosystem of networks within the Internet. From a technical point of view, the interconnection of these subnetworks can be by means of a direct fiber-optic connection, or via one of the 880 Internet Exchanges worldwide.

It is precisely these network gateways that can become the bottlenecks of the Internet if they are not sufficiently upgraded, thus limiting the availability of services in the face of explosive growth in data traffic. The exchange platforms of Internet Exchanges, for example, offer sufficient capacity on a continuous basis and are generally only used at around 50% capacity. Consequently, the connections (i.e. the connected capacities) of the participating subnetworks, which exchange their data traffic at the Internet Exchange, are the limiting factor here. If more data traffic is to flow from all subnetworks connected to the Internet Exchange (e.g. end-user networks) to another subnetwork (e.g. service provider network) than the latter's access capacity at the Internet Exchange, part of the data traffic is inevitably discarded. Typical connection capacities allow transmission rates of 1 Gbps, 10 Gbps, 100 Gbps or even 400 Gbps, which can be increased or combined as desired.

In addition to considering the service provider network itself and the connection capacity at network transitions such as an Internet Exchange, service providers must of course also maintain sufficient server infrastructure to cope with the current increase in usage. It also makes sense to bring the corresponding servers closer to the end customers. Netflix films that are streamed in Germany, for example, are not stored in an American data center, but in a German one.

The world’s largest Internet Exchange as a blueprint for the Covid-19 changes in data traffic

In a more detailed analysis at the DE-CIX Internet Exchange in Frankfurt am Main, as well as at the location DE-CIX Madrid, a breakdown of the data traffic into different service classes – Video on Demand (VoD), Collaborative Working, and Gaming – was carried out, investigating the volume of data traffic and the number of users over a period of several weeks (before and during the restrictive measures caused by Covid-19). This results in three curves, each for maximum, average and minimum data traffic[2] (see Figs. 1.1, 1.2 & 1.3).

Looking at the relative changes in transmitted data traffic (across all service classes) in Frankfurt (Fig. 2), the largest shift occurs in the late evening or very early morning hours: up to 20% more data traffic between midnight and 2am than before the Covid-19 crisis. This suggests that significantly more users can sleep longer in the morning and therefore use the Internet longer in the evening and at night. Another relatively significant change can be observed in the morning and at noon; here traffic increases by up to 18% compared to before Covid-19. This can probably be explained on the one hand by a shift of work to the home office and the associated use of the Internet, including video telephony, and on the other hand by reduced working hours and the resulting extra time for video streaming or online computer games.

The Internet will hold, but for how long?  

Overall, from a technological point of view, the Internet can withstand the onslaught of recent weeks extremely well, both in the end-user networks and especially at the transfer points. Not least because over the past few years there has already been considerable global growth in worldwide data traffic, and the Internet industry has now only had to absorb anticipated short-term growth. One can therefore speak of accelerated digitalization, also with regard to developments in working from home or virtual events. Across the board, people were forced to work from home, which established that working virtually works well and can be integrated more often than before the crisis, if necessary, even beyond the Covid-19 measures. Virtual events are also currently experiencing a great upswing. Although the digital alternatives cannot completely replace personal contact, there are many successful online formats that offer participants a high-quality event and can save them, at the very least, a long journey.

In the medium to long term, there may be challenges for the entire Internet infrastructure, ones that any other industries will also face. Due to the restriction of freedom of movement, maintenance or upgrading of the devices in data centers cannot always take place as planned. This is not a problem in the short term, since on the one hand, sufficient reserves are available at all times, and on the other hand, automation in data centers (e.g. patch robots at DE-CIX in Frankfurt, which can independently upgrade customer capacities) is to some extent quite advanced. 

However, if the current situation continues for further months, the point at which action is urgently needed may be reached. With regard to the supply chains of the hardware used, for example, there could be delays in delivery. Routers, switches or optical equipment is mainly manufactured in the Asia. As these devices are subject to constant physical stress when in use, they will have to be replaced sooner or later, although the typical cycles for this can be measured in years – and a crisis lasting for years, including delayed deliveries, is rather unlikely.

The bottom line: The virus as a driver of digitalization 

One thing is clear from the current situation: The Internet’s holding firm! Even during such phases of maximum load as a global shutdown. However, during the global pandemic, some weaknesses in the digital infrastructure have also emerged, some of which can be felt directly by the user, or will be felt in the long term. Whether it is a question of the challenges in the subnetworks of the Internet, at the network gateways, or in the last mile (the end-user connection); ultimately it is the responsibility of the respective operators to maintain sufficient capacities and to expand them in a forward-looking manner as necessary. Now, in the unexpected current crisis, this has to happen faster than planned and across the board. This in turn greatly accelerates digitalization, with the coronavirus pandemic as an extremely unusual driver. Disruptive technologies, such as 5G, will accelerate the challenges and demands on “the network”. Only if all participants in Internet infrastructure work well in their own way, drive innovation, and show themselves to be unfailingly reliable, will the network of subnetworks, once established – decades ago – for research purposes, be able to meet people’s needs in the future.

Dr. Christoph Dietzel is Global Head of Products and Research at DE-CIX. Previously, he was a Head of the DE-CIX R&D team, and he has been responsible for several research initiatives, including numerous projects funded by the public sector (EU, German Federal Ministries). Chris gained his PhD at the Technische Universität Berlin and has published at various renowned conferences and for journals including ACM Sigcomm, ACM IMC, IEEE Communications Magazine, and IEEE Journal on Selected Areas in Communications.

His ongoing research interests focus on Internet measurements / security, routing, and traffic classification. Chris is also highly interested in IXP-related aspects of the Internet ecosystem.