The CPU Is Dead. Long Live the CPU

The more graphics processing units (GPUs) Nvidia (NASDAQ: NVDA) sells, the more central processing units (CPUs) the datacenter needs.

That was not the original storyline. Starting circa 2024, with the AI boom, the story was that GPUs won, CPUs lost. Parallel math beat general-purpose compute, and the old king of the datacenter, CPUs made mostly by Intel, was pushed aside. Nvidia became the symbol of the age—and the most valuable company in history.

But new abundances routinely create new scarcities.

GPUs have become extraordinarily powerful at the core task of AI: matrix multiplication—the dense, repetitive numerical work required to train and run large models. But, as a recent deep dive in SemiAnalysis argues, the more powerful that core becomes, the more it depends on everything around it.

Models don’t spring from GPU clusters ready to use. They require several complex support functions doing the sort of work for which CPUs were created.

Consider “reinforcement learning,” or RL. Reinforcement learning is essentially learning by doing: allowing a new model to try actions, see outcomes, and receive some kind of reward for success or penalty for failure.

If the model writes code, the system may need to compile the code, run it, test whether it worked, and score the result. If the model is solving math problems, the system may need to verify the answer. If it is learning from a simulation, the system may need to run the simulation itself.

These endeavors can consume enormous CPU resources because they involve lots of branching, checking, compiling, interpreting, and simulating, just where CPUs excel.

A GPU is excellent at running the same mathematical operation over and over across huge quantities of data. A CPU shines when the workload is more irregular, more conditional, more “branchy.” Did the code compile? Did this tool call return the expected format? Does the simulated environment react this way or that way?

Increasingly, the industry wants so-called agentic AI: systems that can use tools, query databases, search the web, and take multiple steps toward a task. Similarly, “retrieval-augmented generation” (RAG) goes beyond the model’s internal parameters, to pull outside information from documents, databases, or other sources. It feeds that material into the model before producing an answer. The model, in effect, reaches outside itself.

These are not the kinds of job GPUs were built for. When an “agent” launches several tool calls, queries a vector database, checks permissions, pulls documents, reformats results, and sends the enriched prompt back into the model, that work falls mostly to general-purpose CPU compute, storage systems, and networking.

Then there are the data pipelines.

A modern GPU cluster can consume astonishing amounts of data. But before the data reaches the GPU, it often must be stored, indexed, shared, moved, decoded, and prepared. Images or video may need to be decoded.

When your super-expensive Nvidia GPU is hanging out waiting for data, you are wasting one of the most valuable assets in the datacenter. The CPU’s job in many AI systems is to keep the GPU fed. And the more GPUs are linked together, the hungrier they get.

For all these reasons, frontier AI labs are increasingly constrained not by GPU availability, but by CPU capacity.

The GPU gets the glory. The CPU does the staff work of the AI age—coordinating, preparing, validating, and feeding every step the model takes.

Intel (NASDAQ: INTC) long dominated the CPU realm. But the investment implications of ramping CPU demand are not as simple as “buy Intel.”

Rising CPU demand does not erase Intel’s structural problems. Advanced Micro Devices continues to take share with stronger designs, and the hyperscalers—Amazon.com (NASDAQ: AMZN), Microsoft (NASDAQ: MSFT), Google’s parent Alphabet (NASDAQ: GOOG)—are increasingly designing their own silicon.

The market is growing. That does not mean every incumbent wins.

Still, the broader point stands.

Success at one layer of a system tends to expose weakness at another. Make the math dramatically faster and it turns out the supporting technologies—software, storage, and general-purpose compute—were not side issues after all. They were simply waiting their turn.