“I cannot yet imagine an application that would need 100 Tbps, but in the future there will be one.“

The Internet we know today would not have been possible 25 years ago. That’s because bandwidth has been constantly growing – in a push and pull between need and potential – for as long as the Internet has been a medium for private and commercial communication. It has always been hard to say what drives what. When larger bandwidths become available, very quickly new applications emerge that exploit it. But existing applications that are underperforming because of a lack of bandwidth also drive the growth of bandwidth (e.g. online video).

How big is big?

Looking back through the history of the Internet, it is clear that what we have available at any one point has always seemed so impressive at the time. In the world of Internet Exchanges (IXs), typified by the story of DE-CIX, milestones have come and gone – with connections of 100 Megabit Ethernet, 1 Gigabit Ethernet (GE), then 10 GE, and a few years later with 100 GE. Initially, such bandwidths seemed thoroughly extravagant, and then just as suddenly it’s a thing of the past. Currently we are at another of these points with 400 GE – which may seem huge today, but Daniel Melzer, Chief Network Architect at DE-CIX, is convinced that in ten years we’ll look back and laugh. As Frank Kammer from Axians says, “I cannot yet imagine an application that would need 100 Terabits per second (Tbps), but in the future there will be one, no question about it. I could not have imagined that our children would actually use 100 Mbit DSL, but they sure do.“

Explosive growth in performance and bandwidth

To put the growth in perspective, data centers in the year 2000 were dimensioned for 500 Watts per m² (= rack). Today, we find dimensions ranging up to 25 KW per rack. This represents 50 times the power consumption of the early Internet data centers, but with even greater growth in performance. Back then, little thought was given to energy efficiency, whereas nowadays we understand that data centers can play a major role in the development of sustainable digitalization. Today, ensuring the greatest output per KW is fundamental to the design of the data center itself and the hardware that goes in it.

Meanwhile, back at the turn of the millennium, 2 Megabits (Mbits) of bandwidth was standard for commercial high-speed connections – and having access to one of these was hugely expensive. Today, companies are connected to the Internet with between 100 Mbits – 50 times more bandwidth – and 1 Gigabit (Gbit) – 500 times the bandwidth – or they have fiber to the building, in which case bandwidth can be much, much greater.

Growing from interconnecting a few to hundreds of customers

Back in the early days of networking, networks were connected using what was called a hub

In the early years, DE-CIX was already working hard to improve Internet performance for its connected customers. The original idea in 1995 was to connect three Internet service providers with one another in Frankfurt, Germany. However, it quickly became clear that there was interest from other networks and the potential for growth. At the turn of the millennium, connected customers stood at 95, with double digit annual growth, while peak traffic was at 4 Gbits and doubling each year. Moving forward, today there are more than 1,000 customers connected in Frankfurt, and they exchange over 10 Tbits of traffic (2,500 times more) at peak times. But despite the unbelievable development of the last couple of decades, we are still, relatively-speaking, taking the first steps in a long-haul journey into the future of connectivity.

Pushing technological boundaries to keep pace with growth in demand

Back in the early days of networking, networks were connected using what was called a hub. A hub is a device where only one of the connected devices could speak at any one time, with all others needing to be quiet. This can work when there are not many connected parties, and when there is not much data traffic being exchanged. But the more parties that are connected, the less feasible such a one-way system becomes. As a result, it wasn’t long before the change was made to using switches to connect networks. With switches, two participants are always interconnected with each other. The larger the switch is, and the more participants that are connected, the more of these one-to-one relationships can occur simultaneously, which means much more data can be transmitted at the same time. This was the cue for the emergence of the Internet Exchange (IX), including the fledgling DE-CIX in 1995.

The success of DE-CIX Frankfurt, and the growth in connected customers and traffic flowing over the exchange, was such that it outpaced the speed of technological development, as Daniel Melzer reminisces. “In the years 2008-2012, we had to swap out our devices to some extent every 6 months because they were simply full. We needed the next, slightly larger, model in order to grow a bit. We had very short cycles, and we had to change the technology regularly because the hardware was not large enough or fast enough for what we needed.”

This constant racing against the clock is what led to Melzer’s conception and design in 2012 of the cutting-edge DE-CIX Apollon platform. DE-CIX needed a solution that would grow with the Internet Exchange.

DE-CIX Apollon – catapulting interconnection into a new era

Going back to the time shortly before DE-CIX Apollon, 2012, DE-CIX Frankfurt was already experiencing close to two Terabits per second of traffic. By this stage, the number of customers and amount of traffic meant that a switch could no longer do the job effectively. The concept “switch”, as years earlier the concept “hub”, no longer represented a feasible solution for the growing exchange.

By this stage, the number of customers and amount of traffic meant that a switch could no longer do the job effectively

Daniel needed to come up with a new solution that would be more sustainable in the long-term, to allow for the growth that DE-CIX anticipated. He was on the look-out for a new way of interconnecting networks, but some of the technology didn’t exist yet, as Frank Kammer from Axians explains: “DE-CIX’s ideas were very, very advanced, and the technology was not yet available. A year later, we had the technology that DE-CIX wanted, and then we got together. DE-CIX is always a customer who demands faster, further, better.”

“In order to support such a high volume of traffic, we actually needed to have multiple things in parallel to achieve the required level of performance.”

As Melzer explains, “We were at the end of the road in terms of switch technology at the time. This led to us looking around in a quite different segment to build something completely new, which raised the concept of the IX to a completely new level – it catapulted us into a new era. We no longer bought normal switches, we bought enormous routers – which had previously only been used by carriers – and in principle, we virtualized the service “IXP” (Internet Exchange Point) on these. This meant that it was no longer one switch for one IXP. In the times of DE-CIX Apollon, we have a platform, and this functions as the basis on which several DE-CIX IXs can run.”

Wim Hendrickx from Nokia reminisces that, “What was important for DE-CIX was that there was a lot of traffic, and they needed support with technology that could scale out very well and be very easily expandable. In order to support such a high volume of traffic, we actually needed to have multiple things in parallel to achieve the required level of performance. And also, with the XRS router, we had a product where we could connect multiple platforms together so they could connect their customers in a way that would scale out. This means that if you run out of port capacity on one platform, you can extend it to the next one.”

Unlike anything seen before

This meant that DE-CIX could take Internet Exchange technology to a new level, with a device class that – based on the number of ports, traffic, and data throughput – was in another dimension entirely to what had previously been in operation. The devices came from Alcatel Lucent (now Nokia), with an underlying optical network based on ADVA technology and supplied by Axians, and this new design was capable both of transporting the enormous data volumes, and of scaling into the next decade. That meant, in principle, that everything needed to be made anew: The hardware was new, the optical network was new, the topology was new. In this combination for an IX with such – in those days – huge data volumes and for the number of connected customers, Apollon was unlike anything seen before.

Taking interconnection to the edge

When DE-CIX Apollon was conceived and built, it was designed for DE-CIX’s flagship Internet Exchange in Frankfurt, primarily to sustain further growth. At that stage, the focus had not yet turned to New York or any of the other (still long in the future) locations. Certainly, thought had been put into how scalable and expandable the DE-CIX Apollon platform would be, and whether further locations could be integrated. But it was only in 2014 that DE-CIX’s international expansion really began – starting with New York, and followed in 2015 with the Mediterranean locations. These locations were initially built as islands and then later connected to one another, meaning that DE-CIX Apollon is the basis also for this large-scale interconnected infrastructure. Today, this infrastructure encompasses a total of 25 IXs, most of which are directly interconnected, accessible from more than 500 data centers in 80 countries, an interconnection ecosystem spanning seas and oceans and continents along DE-CIX’s digital interconnection equator.

International expansion was a significant change in the conceptual path of DE-CIX, taking interconnection services to those areas where our customers have the greatest need, following the traffic flows. Of course, this concept of glocalization – global expansion while getting closer to the user at the local level – will progressively be taken even further, even closer to the edge, together with countless partners around the world, as the DE-CIX global interconnection ecosystem continues to grow.

Network automation and quantum communication – building the network of the future

In principle, current technological concepts are still rooted in a time where the standard was to connect things using cable, and if more bandwidth is needed, then a special technique is used to get more out of the cable, or more cable is added. Everything is dependent on this hardware and this connection.

“They ease the pain, but I think this is the pain of a foundering technology.”

In this world, interconnection will become more and more important, and the infrastructure for interconnecting networks will become the foundation of the economy and industry

Melzer sees the whole topic of cabling and the employment of patch robots as being part of a transitional phase. “They ease the pain, but I think this is the pain of a foundering technology. The time is right for something different. The problem is that they can’t really be sensibly automated, because someone needs to be on location to build them. I would estimate that this is the biggest inhibitor to being able to build a network that is simply out of this world.  The state-of-the-art of Internet technology today – as out there and awesome as it is – is really right at the top of the physical limit. We’re squeezing that last little bit we can get out of it, and we can actually only grow through going parallel. I can really imagine that something will be developed in the future that is at least as disruptive as the Internet was in comparison to traditional telephony. Whether that will be quantum entanglement, quantum communication, or whatever, the past has shown us that things will be developed that turn everything upside down. I expect that, and I’m waiting to see if it will happen in my lifetime.”

Interconnection infrastructure that connects everything

Today we have the beginnings of an interconnection infrastructure that connects everything. In the future there will be a much more intricate mesh of data suppliers, data processors, and data users, with

1. the rollout of 5G & the development of large-scale IoT projects for cities, industry, mobility, and the healthcare sector,

2. increasing levels of artificial intelligence and machine learning in the handling of data, and

3. the increasing number of points where data is collected and processed and the enormous amounts of data generated, collected, summarized, and sent somewhere to do something new with it.

In this world, interconnection will become more and more important, and the infrastructure for interconnecting networks will become the foundation of the economy and industry – as essential to future life and business as power and water lines are today. DE-CIX, supported by our trusted hardware partners – without whom we would not even be here – and together with our many partners from across the digital ecosystem, will be there, in the future – as we have been in the past.

More stories:

• The “WOW” effect: How helping customers has that winning feeling - Interview with Wim Henderickx, Director of Network Consulting Engineering and PLN, Nokia
• A real pioneering time - Interview with Frank Kammer, Business Area Manager Telekom & Carrier, Axians Germany
• The Internet: not the same without...