Most of the AI infrastructure conversation over the last few years has centered on one thing: the accelerator. How many GPUs, how much HBM, how fast the scale-up fabric. That focus made sense when most infrastructure planning centered on training and model serving. Agentic AI broadens the workload, and it’s starting to change what the rack needs to look like.

An agent doesn’t just call a model. It plans a task, writes and runs code, calls tools and APIs, queries a database, checks its work and tries again. The model call is one step in that loop. Everything around it runs on general-purpose compute. And as agents move from pilots into production, that surrounding work grows right along with them.

AMD clearly sees this. This week the company introduced Helion, an open rack blueprint for agentic AI built on Open Compute Project standards and powered by 6th Gen AMD EPYC processors. It’s a rack designed around what agents actually do all day, not just around the model they call. And it’s one of the clearer attempts I’ve seen to design the rack around the full agentic workflow rather than just the accelerator.

What AMD Announced

Helion is a rackscale reference design, not a single fixed product. AMD’s OEM and ODM partners build and sell the systems. The blueprint comes in two base configurations:

• Hyperscale configuration. A CPU-only rack for hyperscalers, neoclouds and sovereign AI operators that run agents at very large scale. It supports up to 32,768 “Zen 6” cores and is built to optimize agents per watt. These customers already run separate inference infrastructure, so the rack focuses entirely on agent execution.
• Enterprise configuration. An all-in-one rack that mixes EPYC CPU blades, AMD Instinct MI350P GPU blades and general-purpose compute in a single system. It includes a token router that sends agents’ model calls either to frontier models in the cloud or to open-source models running inside the same rack.

Both configurations use a 21-inch ORv3 rack with direct liquid cooling and air-cooled options. The rack supports up to 102 Tbps of networking bandwidth. AMD plans to contribute the Helion spec to OCP, and partner configurations are expected in November 2026 from the likes of Cisco, Dell, HPE, Lenovo, Supermicro, QCT, Wiwynn and Hon Hai.

Not All Agents Are the Same

The part of this launch I like most is how AMD segments the workload. It doesn’t treat “agentic AI” as one thing. AMD groups agent workloads into three profiles, each with a different optimization target:

• Personal agents are about maximum agents per watt. Think high volumes of lightweight, always-on assistants. AMD pairs these with dense servers running 256 cores at 400 watts.
• Enterprise agents are about running the maximum number of high-performing agents. These are agents working through business workflows and enterprise applications. AMD pairs them with balanced servers running 256 cores at 600 watts.
• Technical agents are about maximum performance per agent. These are agents doing compute-heavy, latency-sensitive work. AMD pairs them with performance servers running 96 cores at up to 5.0 GHz.

Why does this matter? Because it reflects how agentic AI will actually look in an organization. A company might have thousands of lightweight agents handling routine requests, a smaller fleet of agents running finance or supply chain workflows, and a handful of engineering agents doing simulation or code analysis. Those are very different compute profiles. A rack that lets you mix and match blades to fit that mix is a rack that wastes less power and less money.

It’s the same lesson the industry learned with general-purpose servers years ago. One size rarely fits all. The difference now is how quickly the mix can shift as organizations move from simple assistants to workflow and technical agents. Infrastructure teams need room to adjust as it does.

Two Racks for Two Different Buyers

The split between hyperscale and enterprise configurations is also smart. For me, it shows AMD understands its buyers.

Hyperscalers already operate massive, separate accelerated infrastructure. For them, agent execution is its own tier, and the job is packing as many agents as possible into every watt. For that operating model, a CPU-only rack can be the more efficient answer.

Enterprises are different. Most IT organizations aren’t going to stand up one tier for agent execution, another for inference and a third for the databases and storage those agents depend on. They want one platform they can deploy, manage, secure and power within the datacenter they already have. That’s what the enterprise configuration delivers. CPU blades handle agent execution. GPU blades handle inference. General-purpose blades handle the database, storage and technical computing work that agents lean on to get anything done.

The GPU piece is sized for that purpose. The enterprise configuration supports up to two MI350P servers, each with eight GPUs in 5U, for 16 GPUs per rack. AMD is clear that the two nodes run independently, with no scale-up GPU fabric between them. That’s a deliberate choice. This rack is built for enterprise agents and inference, not frontier model training. For that, AMD points customers to Helios, its 72-GPU rackscale platform. Helion and Helios serve different needs, and I think keeping that line clear will help customers.

The air-cooled option deserves a mention, too. If you’ve ever run an enterprise datacenter, you know liquid cooling isn’t a switch you flip. Plenty of facilities can’t support it without major work. An agentic AI rack that fits into existing air-cooled space lowers the barrier to getting started. Again, this tells me that AMD has really thought through how these systems land in the wild. Liquid cooled is fantastic at delivering efficiency. Unfortunately, many enterprise datacenters were built a decade or more ago, and plumbing for what’s new isn’t easy – or possible in some cases.

Open at Every Layer

“Open” gets used loosely in this industry, so I appreciate that AMD spells out what it means here. Helion implements OCP standards across the rack:

• ORv3 for the rack and busbar
• DC-MHS for the host processor module
• DC-SCM for management and security
• HPRv2 for power shelves
• OCP NIC 3.0 for networking
• Caliptra for the silicon root of trust

That’s an OCP-based design across multiple layers of the stack, not just a rack frame that happens to fit OCP dimensions. And it matters for a practical reason. Open, modular building blocks give OEMs room to differentiate on integration, cooling, services and support, while giving customers real choice in who builds and supports the system. Enterprises can buy Helion-based infrastructure from the vendor they already trust, and that vendor can tailor it to their environment.

The security pieces are worth calling out for enterprise buyers specifically. Agents act on behalf of users and organizations. They touch sensitive data and take actions inside business systems. So, a hardware root of trust and standardized management and security modules aren’t nice-to-haves in that world. They’re table stakes.

The Token Router Is the Sleeper Feature

The enterprise configuration includes something that hasn’t gotten much attention yet: an integrated token router that directs agents’ model calls across frontier models in the cloud and open-source models running in the rack.

I think this could turn out to be one of the more important parts of the launch. Every model call an agent makes is a decision about cost, latency, data governance and capability. Does this request need a frontier model? Can an open model running locally handle it for a fraction of the cost? Should this data leave the building at all? Today, many organizations make those calls inside the application layer, if they make them deliberately at all.

Putting routing capability in the rack brings some of that decision-making closer to the infrastructure layer. And as agents multiply, token volume can grow much faster than user counts. Routing those tokens intelligently could be a big lever on AI economics.

With this said, I haven’t seen much detail on how the router works, how it’s managed or how it integrates with the AI gateways and governance tools enterprises are already putting in place. I’ll be looking for those answers as partner systems come to market.

What This Means for Enterprise IT

So what should IT organizations take from this? I see three takeaways.

Plan for the whole agent, not just the model. Most AI capacity planning so far has started and ended with GPUs. Agentic AI changes that. Agents consume CPU cores, memory, networking and storage between every model call. If you size only for inference, you risk leaving expensive accelerators underused while the rest of the workflow becomes the bottleneck. Start modeling the CPU-to-GPU ratio your agent workloads actually need.

Match infrastructure to the agent profile. Not every agent needs the same compute. Lightweight, high-volume agents reward density and efficiency. Business-critical workflow agents reward balanced performance. Technical agents reward clock speed. As your agent portfolio grows, look for infrastructure that lets you tune the mix rather than lock into one configuration.

Treat model routing as a governance decision. Which models your agents call, and where those models run, affects cost, latency, data residency and compliance. Whether routing lives in the rack, in an AI gateway or in the application, someone in IT needs to own it. Decide now who that is.

What I’m Watching

A few things I’ll be tracking as Helion moves toward availability:

• Partner systems. November’s right around the corner. I want to see how Dell, HPE, Lenovo, Cisco and others package Helion, and how they differentiate on management, services and support.
• The token router. How it works in practice, how it’s managed and how it fits alongside existing AI gateways and governance tools.
• Real-world ratios. Where early enterprise deployments land on the balance of CPU, GPU and general-purpose blades. That data will tell us a lot about how agentic workloads behave in production.
• The OCP contribution. How quickly the spec moves through OCP, and whether the broader community builds on it.

The Bigger Picture

AI infrastructure has spent the last few years organized around the accelerator. Agentic AI is broadening that view. Agents need models, but they also need a lot of general-purpose compute, storage, networking and security around those models. The rack has to be designed for the whole workflow.

Helion is AMD’s answer to that shift. It’s open, it’s flexible, and it reflects a clear understanding that hyperscalers and enterprises need different things. AMD also frames Helion around making efficient use of available power, and that’s the right lens. Power is increasingly one of the defining constraints on AI growth, and every watt spent on an agent needs to deliver useful work.

For enterprise IT, the lesson goes beyond any one vendor. Agentic AI is a whole-system workload. The organizations that plan for it that way will get more out of every rack, every watt and every dollar.