What is direct-to-chip cooling for data centers?

As AI continues to advance, data centers don’t just need to evolve — they need to evolve fast.

Handling a surge in power demand is one concern. But heat is another pressure. AI data centers create vast amounts of it, with recent — not yet peer-reviewed — research even suggesting that they can create ‘heat islands’ that increase the temperature of the land around them.

At the heart of the issue are rack densities — the power consumption and heat generation of the IT equipment within a single server cabinet — and traditional cooling technology that is being pushed to its limit.

How are data centers cooled?

Traditional data center cooling systems, which rely primarily on air conditioning, usually handle IT loads of around 20 kilowatts (kW) per rack.

But AI workloads, driven by heavy clusters of high-performance processors such as GPUs, currently generate rack densities of 100kW. This is projected to rise to 1.2 megawatts (MW) by 2028.

This is forcing a move to liquid cooling, which can support much higher rack densities, to over 200kW, because liquids can absorb and transfer heat more efficiently. As such, the data center liquid cooling market is projected to grow from $6.6 billion this year to $38.4 billion by 2033.

There are two main types of liquid cooling used in data centers: immersion cooling, which submerges entire servers in thermally conductive liquid; and direct-to-chip cooling.

What is direct-to-chip liquid cooling?

Direct-to-chip liquid cooling is widely seen in the industry as an effective and scalable technology for handling the heat of AI workloads.

The technology sends coolant through cold plates mounted on the hottest components, especially CPUs and GPUs, meaning heat is removed close to the source rather than being absorbed into the air.

It saves space, enables components to run faster, and uses less energy than fans used for air cooling, which is significant when you consider up to 40% of data center electricity use goes to cooling.

What is two-phase direct-to-chip liquid cooling (DLC)?

There are two types of direct-to-chip cooling: single-phase and two-phase.

In single-phase cooling, the liquid coolant is moved through the cold plate to cool the chip it is mounted to. The heated liquid is taken away using pipes, and then a dissipation mechanism — such as a fan — is used to disperse the heat. It is a closed-loop system, so the liquid is then fed back for the process to repeat.

As thermal design power continues to rise, requiring cooling systems to remove higher levels of heat from CPUs and GPUs, the data center industry is looking for more advanced solutions.

Two-phase DLC, meanwhile, takes this a step further — expelling the heat by evaporating the refrigerant in the cold plate, removing significantly more heat than just raising the liquid’s temperature. The vapor is then condensed back into liquid in a heat rejection unit and pumped back into the system to repeat the cycle.

What are the benefits of two-phase DLC?

This process, which has a number of benefits over other types of data center cooling.

Mitsubishi Heavy Industries (MHI) — which has applied two-phase DLC — says the heat transfer rate is an order of magnitude higher than conventional air cooling, which enables stable GPU operation while reducing the failure risk for the entire system.

MHI’s two-phase DLC uses non-conductive, electrically insulating, refrigerants. No water is used, so if a leak occurs, it will not short the electrical circuits, preventing damage to equipment.

And two-phase DLC also uses less power because it removes heat more efficiently, meaning fans and other cooling equipment do not need to work as hard. This can bring cost and emissions benefits.

What’s next for two-phase DLC?

Analysts expect two-phase DLC to enter large-scale deployment within the next few years.

MHI is testing how the technology can be employed to enhance the performance of existing data centers in Japan, without major modifications.

And, working with infrastructure construction company EXEO Group, it has started the first commercial use in Japan of GPU servers with two-phase DLC. The companies aim to combine MHI’s heating and cooling expertise and EXEO’s experience in building and operating data centers to create a “one-stop” service for GPU servers.

Cooling may have become a strategic bottleneck for data center performance – but the latest technology points to an efficient future as the data center market continues to heat up.