Enterprises still lack a real, actionable way to compare agent performance across models, infrastructure and vendors. Without that, they cannot say with confidence what to deploy or what it will cost to run at scale, and that gap is slowing adoption and undermining any defensible ROI case for agentic AI. Signal65’s new Pinnacle benchmark, launched earlier this month, is the first methodology I have seen that scores agents on whether they get the work right, not on how fast or how cheaply a model responds.

Why Current Agentic AI Benchmarks Miss The Point

1. Model and GPU benchmarks only tell part of the story. Business results come from agents, and agents rely on models, tools, harnesses and infrastructure, each of which can swing the score.
2. Agents and the humans running them are diverse. How an agent gets used, and with what context, produces inconsistent results, so real testing has to account for different user personas and software harnesses.
3. Vendors tune their own testing configurations to flatter their products, producing biased scores that will not hold up in the real world.
4. Most importantly, benchmarks measure model speed or response latency more than outcomes. Getting to the wrong answer faster does not help anyone. Getting to the right outcome does.

Signal65’s answer is Pinnacle, the agent benchmarking methodology it launched earlier this month. Pinnacle reads every test run through three lenses: Model Intelligence, which grades the model’s output for quality and correctness; Silicon Capacity, which measures how the underlying infrastructure holds up under real agentic load; and Solution Economics, which prices out the cost per correct task. Model Intelligence alone already beats biased, vendor-run testing. Stacking the infrastructure and cost lenses on top gives enterprises a clearer picture of what works and what it will actually cost to run.

What Pinnacle Changes For Enterprise Deployment Choices

Signal65 has been running projects across all three lenses for a while with agent-based products. I collaborated with the firm on Amazon Bedrock AgentCore testing last year, which fed into a research paper Patrick Moorhead and I co-authored at the same time. Pinnacle reflects that past work and improves on it in three ways.

1. It removes the barriers of bespoke testing. Pinnacle reflects work Signal65 had already done, and bespoke client engagements are costly and, given how fast agents evolve, short-lived. Signal65 is investing to remove a barrier that has held back agentic benchmarking for roughly two years.
2. It accounts for multiple user personas. We consistently find that treating every agent user as a single archetype produces inconsistent results. A salesperson uses agents differently than someone in finance or HR, and Pinnacle’s testing reflects that.
3. It is built to eliminate vendor bias. An independent third-party team runs Pinnacle using a consistent, repeatable set of tools, and vendors get no veto over results. That is Signal65’s design intent; based on the governance model behind it, I would call the result bias-resistant rather than bias-proof, which still clears a meaningfully higher bar than most vendor-funded testing.

What Pinnacle Still Needs To Prove

Pinnacle marks real progress on transparency and usefulness, and the reason nothing like it existed before now is the scale of investment required to build it properly. Signal65 will be the first to say this is a new approach that will keep evolving. Two challenges stand out.

1. Keeping pace with the technology itself. Pinnacle’s Silicon Capacity lens currently covers NVIDIA and AMD GPUs, a reasonable starting point given where most enterprise agentic workloads run today. But that is not the whole map. Google’s TPUs are showing up in more production deployments, and if Pinnacle wants to stay the independent reference point for infrastructure choice, its coverage needs to grow with the market rather than chase it. This is less a knock on Signal65 than a reminder of the treadmill every credible benchmark runs on: stop adding silicon and frameworks to the test matrix, and the benchmark becomes a snapshot of a market that has already moved on.
2. A perception problem, not a product problem. I still hear a version of the objection SaaS vendors are fighting right now: why pay for independent testing when you can just ask Claude, ChatGPT or Gemini for the answer? That framing skips the actual work. Pinnacle’s value is not the model call, it is the controlled, repeatable, occupationally grounded test harness and governance behind it. Signal65 has to keep making that harness visible rather than assumed. The more it publishes results and opens its methodology to scrutiny, the easier it becomes for enterprises to see the difference between a chatbot’s guess and a governed benchmark.

Where Pinnacle Goes From Here

The data set Signal65 is building from Pinnacle and other agent-related engagements could become a meaningful source of context for automating benchmarking itself. Two use cases stand out: for vendors, that data could support a more automated, digital-twin-like process for simulating and testing future solutions before they ship. For enterprises, I can see baseline tests adjusted to align with company-specific workflows rather than generic personas.

Pinnacle will not settle every argument about how to benchmark agentic AI, but it is the first methodology I have seen serious enough for enterprises to put on the evaluation checklist now, rather than wait for a second version to mature.