The structural tension at the heart of AI data center growth

AI workloads are forecast to represent half of all data center capacity by 2030, compared to around 25% today. This is expected to drive investment in the data center infrastructure that supports the technology to the tune of up to trillions of dollars by the end of the decade.

As this build-out continues, a concurrent shift is taking place.

Training AI models has so far dominated the need for more data center capacity. But in the next five years, analysts expect inference — the “doing” part of AI, as Google explains it — to surpass training to become the main workload in AI data centers. It will represent more than half of all AI compute and roughly 30-40% of total data center demand by 2030, McKinsey predicts.

At the same time, geopolitical ambitions and sustainability imperatives are intensifying. These converging trends will increasingly shape how hyperscalers plan facilities, experts say, including driving a need for more decentralized ‘edge’ infrastructure.

The shift to the edge

The edge is “where the AI revolution will be won”, according to tech market analyst Omdia.

At a recent technology trends event in Japan, Fiona Kyle, the organization’s Vice President of Research, Integrated Tech, positioned the “paradox of compute” — where investment is being channeled into maturing generative AI while physical AI is emerging — at the heart of the structural forces reshaping the sector.

The shift is driving a fundamental change in “where compute happens, how it scales and who controls it”, and this is changing everything about infrastructure architecture, according to Kyle.

Consumers and businesses don’t want AI as a feature, they want the outcomes it delivers — and AI enabling these improvements is where real value accrues, she added.

Physical AI arrives

Increasingly, billions of smartphones, PCs and wearables are being reimagined with features including live translation and voice assistants.

Physical AI — which enables autonomous systems including cameras, industrial robots and self-driving cars to undertake complex tasks in the real world — is unlocking new capabilities across industries.

One sector that could be transformed by this is manufacturing, which today faces challenges including rising costs, workforce shortages and changing customer expectations.

Technology such as industrial robots that can perform a variety of jobs rather than fixed, repetitive tasks, or learn from and adapt to real-world experiences, promises to boost efficiency and support greater flexibility and resilience on the factory floor, according to a recent World Economic Forum white paper on the topic.

And while just 5% of firms told a Deloitte survey that physical AI is transforming their industry today, more than 40% said they expect it to within three years.

Edge data center innovation

As business leaders look at how to integrate physical AI into their operations, the data center infrastructure that will support it is evolving rapidly.

Alongside continued expansion of centralized training infrastructure, AI’s physical future will increasingly require compute to happen closer to the user at the network edge, because performing such tasks requires ultra-low latency, real-time responsiveness and, often, increased data security.

This means inference will take place on the device itself, in edge data centers, or a combination of both.

In response, edge data center innovation is advancing quickly. Companies in the space are working on modular solutions that can be brought online quickly, advanced cooling technologies to deal with the heat intensity of AI workloads, and using AI to monitor and optimize systems.

Mitsubishi Heavy Industries (MHI) Group, for example, has launched DIAVAULT to accelerate digital transformation in the industrial sector. This platform provides integrated on-premises power, cooling systems and data processing, with levels of high security and low latency that are hard to realize in cloud environments.

The platform provides edge solutions including inference environments in manufacturing sites and research facilities, and 5G connectivity is expected to meet demands for low latency.

1 billion robots

While AI is shifting to the edge, there are other tensions at the heart of the race to develop data centers, as Omdia’s Kyle noted. Technology leaders are being forced to reconcile the speed of innovation with sustainability constraints and geopolitical sovereignty demands.

And while this creates challenges, it also presents opportunities. Physical AI is already here, but it is set to explode over the next decade, with more than 1 billion AI robots forecast to be moving among us by 2035.

Forward-thinking companies — with the right infrastructure supporting them — stand to see resilience, compliance, cost and efficiency benefits.