How AI is driving the rise of edge networks
Our continued demand for generative AI is fueling the emergence of a new type of networking model: ‘edge networking’. We are now witnessing the global development of edge data centres that are placed closer to the end user to meet the real-time responsiveness and low-latency performance requirements of GenAI. To put this into numbers, analyst firm IDC forecasts that spending on edge computing is expected to reach $232 billion in 2024, up 15.4% from 2023.
Edge data centers are small data centers and computing locations that are part of a distributed network of data centers. They are located near the areas they serve, significantly reducing latency and improving the performance of applications that require real-time data processing. This decentralized approach also helps balance loads, ensures data flow in the event of an outage, and improves the overall resilience of a network.
Chief Technology and Operations Officer, Neos Networks.
GenAI Support
The case for edge networking is clear. AI applications are both data-intensive and compute-intensive. However, edge computing promises to overcome these technical challenges by enabling real-time decision-making with reduced latency, local data storage and processing, and reduced data transfer to the cloud. This is particularly relevant when it comes to the need for inference and localized data processing.
With GenAI requiring even faster processing, there will be many existing and emerging applications where networks will need to deliver ultra-low latency. The more time-critical the application, the more data will need to be stored and processed at the edge. Take AI inferencing (using an AI model to draw conclusions from new information or data) for example.
Computing at the edge can reduce the time to a result from a few seconds to a fraction of a second. In addition, several other emerging industrial use cases highlight the need to place compute close to the end user – whether that’s content generation applications like ChatGPT, interactive customer service agents, immersive AR experiences, smart healthcare and smart retail, and predictive maintenance. In these millisecond-sensitive scenarios, the user will enjoy a higher quality experience if the compute is hosted as close to them as possible.
The sustainability argument
A recent TechRadarPro article argued that we don’t have the power to handle the current explosion in demand for data centers, due to AI. That’s why we need to build data centers away from central locations, at the edge. According to Goldman Sachs, a ChatGPT request requires nearly 10 times as much electricity to process as a Google search. Despite the inevitable spike in electricity expenditure from GenAI, edge data centers offer the advantage of reducing grid power consumption at central locations. By distributing computing load across the network, power demand is spread out, not concentrated. By running applications at the edge, data can be processed and stored closer to end user devices, rather than relying on data centers hundreds of miles away.
Investing in an AI-ready network
Investments in high-speed connectivity will make connecting edge sites in the network more practical and sustainable. Fiber optic cables offer significantly lower latency and higher bandwidth than traditional copper cables. This allows for faster data transfer speeds. High-speed fiber networks are easily scalable, so as data demand grows, additional bandwidth can be delivered without significant infrastructure changes. Fiber networks also consume less power than traditional infrastructure, contributing to lower operational costs and a smaller carbon footprint. With advances in pluggable optical technology, the same economic, sustainability, and technological benefits of fiber are now being delivered in the data center.
While projects like the UK’s Project Gigabit and the US’s Broadband Equity, Access and Deployment (BEAD) programme are necessary steps in the right direction, governments must prioritise building out the network edge and better connecting data centres – not just extending fibre to the home (FTTH).
The Key to Unlocking AI Success
As countries race to become leaders in AI, empowering start-ups and defining regulatory parameters are at the top of the agenda. However, AI’s success will depend on a country’s solid network infrastructure and ability to transport significant amounts of data with little to no latency. If networks can’t handle the influx of traffic generated by ‘always on’ Large Language Models (LLMs), AI ambitions could fail.
Therefore, the respective AI strategies should focus on the size, location and quality of the underlying network infrastructure. Despite widespread investments in ‘traditional’ data centers that are having a snowball effect worldwide, such as Google’s new $1 billion data center in the UK announced earlier this year and Microsoft’s AUD $5 billion investment in building data centers in Australia, there is less focus on edge data centers. To meet AI demands, data center build-out needs to be complemented by edge build-out.
A hybrid model?
A hybrid approach of strategically placed data centers at the edge of the network, combined with central data centers, will be essential to manage the rapid flow of information cost-effectively and sustainably. This is particularly crucial for AI inferencing, where data flows to the edge for processing and then back to the core data centers for distribution. Time-critical applications will be better served closer to the edge of the network, but data-intensive and less time-critical applications will be better served in central data centers.
With large companies like Microsoft setting ambitious goals to triple their data center capacity within the next year to deliver AI, we must also consider edge data centers as part of the strategy. Not only to meet the low latency requirements of GenAI applications, but also to take the power pressure off the central grid.
We list the best AI tools.
This article was produced as part of TechRadarPro’s Expert Insights channel, where we showcase the best and brightest minds in the technology sector today. The views expressed here are those of the author and do not necessarily represent those of TechRadarPro or Future plc. If you’re interested in contributing, you can read more here:
