Microsoft’s Homegrown AI Chip Effort Shows Signs of Life After Slow Start

Microsoft is planning to significantly increase production of its internally designed next-generation AI chips next year in hopes of persuading big cloud customers like Anthropic to use them, according to two people with direct knowledge of the plans.

That’s despite slow uptake of the current generation of Microsoft-designed chips, known as the Maia 200. Microsoft plans to publicly unveil its new Maia 300 chip this fall, potentially as soon as next month, one of the people said. The cloud giant has been in talks with chipmaker Taiwan Semiconductor Manufacturing Co. to secure manufacturing capacity for over 300,000 of the chips for delivery in 2027, the second person said—an order of magnitude above the tens of thousands of Maia 200 chips Microsoft has produced so far.

Microsoft ultimately wants to secure capacity for more than a million Maia 300 chips but may be constrained by component supplies and its ongoing capacity negotiations with TSMC, the person said.

Getting Maia off the ground is a key plank in Microsoft’s effort to reduce its reliance on Nvidia chips, a top priority of CEO Satya Nadella. Past efforts have run into roadblocks; the Maia 200 chip was delayed last year after early tests didn’t meet internal goals, and since then the chip has only been used in a small number of Microsoft data centers.

Andrew Wall, Microsoft’s general manager for Azure Maia, declined in a statement to comment on specific production plans but said Microsoft ultimately wants to produce gigawatts’ worth of Maia chips. Wall did not specify a number or time frame, but data centers that require multiple gigawatts of power would typically house millions of AI chips.

“Microsoft continues to invest in custom silicon as part of our long-term AI infrastructure strategy. While we don’t share production volumes, the figures reported don’t reflect the scale of our program. We expect our Azure Maia deployments to support AI workload demand measured in gigawatts,” Wall said.

Microsoft lags behind rivals such as Google and Amazon in its efforts to get its AI chips off the ground. Google and Amazon have won big customer names to use their tensor processing units and Trainium chips, respectively, in their cloud arms—and Google has begun selling its TPUs to customers for use outside its own data centers. In contrast, Microsoft has so far been the only user of its own Maia chips, which it uses to run OpenAI models and its own in-house MAI models, which power its Copilot AI software. Microsoft still primarily relies on Nvidia chips to run most of that software, however.

But Microsoft is confident it can convince more customers to rent its forthcoming Maia 300 chips, potentially including Anthropic, at least partly because of the cost savings they offer. Anthropic has for months been in talks with Microsoft to use Maia chips in the future, The Information previously reported.

Nadella said in June that two data centers were using Maia 200 and the company was planning to scale it to more, including some based internationally. As of late last month, Microsoft was still using it in only two data centers, both located inside the U.S., according to someone with knowledge of the situation.

Even if Microsoft can’t land a big customer for Maia 300, its backup plan is to shift more of its internal AI use to Maia while it continues to rent out pricey Nvidia chips to its Azure cloud customers. Microsoft told investors last month that its Maia 200 chips are 30% to 40% cheaper to operate than cutting-edge Nvidia chips when running OpenAI and Microsoft models; the company has found internally that Maia 300, which is designed to optimize for its models, performs even better, one of the people said.

The order volume Microsoft is considering for Maia 300 is well below the number of chips rivals like Google and Amazon order annually. By contrast, Google planned to produce more than 3 million of its TPU chips this year and 5 million next year, according to Morgan Stanley estimates.

Meanwhile, Microsoft has recently shown more traction for a different line of chips, its in-house Cobalt central processing units. CPUs are a more traditional kind of processor than Maia and other AI-specific chips that compete with Nvidia’s graphics processing units, but demand for CPUs is also now booming. Microsoft said last week that OpenAI, Adobe, and other big customers were using Cobalt in more than 25 data centers across the globe.

Nadella has in recent years privately decried Nvidia’s stranglehold on the AI chip market. In a 2022 email later published in court documents, Nadella lamented that “right now we are a very thin layer on top of Nvidia and all the IP is with OpenAI” and that an unnamed unit within Microsoft would “lose 4 bil next year.”

While it’s not clear exactly which unit incurred the losses, someone with knowledge of the matter said Nadella was referring to the high costs of running OpenAI models on Azure, in part because Microsoft didn’t have control over the cost of chips or of the AI models themselves—two areas its homegrown AI efforts have since aimed to address.