REDMOND, Wash. — About 5 minutes before each hour, and 5 minutes after it, Microsoft Teams gets busy with people joining meetings early or late. Yes, you know who you are.
Those predictable spikes in demand are an example of a larger infrastructure challenge facing the industry: either build data centers full of chips and servers that sit idle for much of the time, waiting to handle peak loads, or risk performance issues during moments of high demand.
Microsoft has come up with a different approach. The company said Tuesday that it has successfully prototyped new advances in the field of microfluidics that let its servers run hotter and faster — a technique known as overclocking — without the risk of melting down the silicon.
“We’ve got these really spiky workloads — I would dearly love to be able to overclock … because then we would need way fewer servers and deliver a better experience,” said Jim Kleewein, a Microsoft technical fellow and development director for core Office 365 services, recounting the challenge he originally brought to Microsoft’s silicon development teams.
Microfluidics overall is not a new concept. Instead of placing a cooling plate on top of a chip, the approach brings liquid coolant directly inside the processor itself through tiny channels.
But Microsoft says it has developed a new approach that uses AI to customize the cooling system for the unique heat signatures of different chips running specific workloads.
The design, inspired by biology, ends up resembling the veins in a leaf, delivering coolant with greater precision to a chip’s hottest spots. The company says it’s up to three times more effective at removing heat than current cold plates, making overclocking more feasible.
‘Surface of the sun’
Some of the hot spots on a chip “could have the same heat flux as the surface of the sun when you look at the very small scale,” said Husam Alissa, Microsoft’s director of systems technology, referring to a technical measure of heat intensity, or density in a specific area.
While the working prototype was demonstrated on a commercially available Intel Xeon chip, Microsoft says the innovation is part of a broader strategy to improve its entire hardware fleet. As a next step, the company is looking to incorporate the cooling technology into future versions of its own silicon, potentially including its Azure Cobalt chip and Maia AI accelerator.
Those first-party chips, unveiled in 2023, reflect Microsoft’s efforts to gain control over the final piece in its cloud platform. The company is competing to efficiently train and run cutting-edge AI models against Amazon, Google, and others, which are also making their own chips.
Microsoft still uses third-party chips, including industry-standard GPUs from Nvidia and other suppliers, but it says a more integrated approach is necessary to meet the challenges of the AI era. The goal isn’t to replace partners but to advance the broader ecosystem, said Rani Borkar, the corporate vice president who leads Azure hardware systems and infrastructure.
“The demands of Al are such that, frankly, hardware on its own cannot keep up,” Borkar said during a recent media briefing and tour of Microsoft’s Silicon Lab in Redmond. “We really are all about co-designing, co-optimizing every layer of the stack.”
Networking and steel
The company made two related announcements Tuesday morning:
- Microsoft is working with Corning and Heraeus to move to industrial scale production of its hollow core fiber technology, which transmits pulses of light through a hollow channel vs. a conventional solid glass core, letting data move faster and with lower latency.
- In an initiative focused on sustainability, Microsoft said it will procure “green steel” from partner Stegra for use in datacenter construction. The steel is produced in a plant designed to reduce carbon emissions by up to 95% vs. traditional steelmaking.
As for the newly announced microfluidics advances, Microsoft says it will work with partners and the broader tech community to make the technology an industry standard.
Longer term, the company sees the cooling technology as a foundational step toward a revolutionary new type of chip architecture: 3D stacking. While stacking layers of silicon dramatically reduces latency by shortening the distance data has to travel, it has been largely unfeasible because of the immense challenge of removing heat from the inner layers.
Microfluidics could eventually solve that by allowing coolant to flow between each layer of the silicon brick, a breakthrough that Microsoft’s Kleewein believes could be transformative.
“That is where this could move from, ‘it’s an interesting change that the rest of the industry should adopt,’ to a ‘holy shit’ moment in the evolution of technology,” he said.
