Next-Generation Thin-and-Light Laptops Require More Memory Bandwidth and Better Approaches to Thermal Management
As token volumes and cloud costs rapidly compound thanks to more agentic workloads, power users of AI are buying their own local inference hardware, and the PC industry is responding. This is helping to shift more AI inferencing to the edge. In particular, local inference on thin-and-light laptops is projected to become a core target for the industry, based on vendors’ product roadmaps. The challenge is that these machines are coming up against hard physical limits — especially for thermal management — as they address the demands of local inference in an already space-constrained form factor.
Tokens per second is the performance metric that matters for local inference, and it mostly scales with memory bandwidth. Memory bandwidth, in turn, is a function of bus width, capacity, and speed. Most laptops today use 128-bit memory buses paired with LPDDR5, which caps bandwidth at roughly 150 GB/s. But that is not enough to run today’s frontier-quality models at acceptable speeds.
This reality has brought the industry to an inflection point where memory and the associated functions to accommodate it must be improved. Chip designers are already taking steps to achieve this. Qualcomm has moved to 192-bit bus widths in its X2 Elite family. AMD and NVIDIA have 256-bit offerings today. Apple’s M5 Max processor uses a 512-bit bus that operates at 614 GB/s, and is capable of running a high-end model like gpt-oss-120b entirely locally. All available evidence indicates that the next 18 months will bring a broader shift to 256-bit and wider architectures, paired with the introduction of LPDDR6, which delivers faster data rates, better power efficiency, and improved error correction. The confluence of LPDDR6 and improved bus widths capable of robustly supporting local inference is expected to become mainstream in 2027 and 2028.
The chip suppliers are doing their part. To meet the growing demand for local inference in notebooks, OEMs must now solve the spatial challenges that come with wider buses. A 256-bit interface requires about eight length-matched, shielded 32-bit channels with traces under 25mm, so memory must sit adjacent to the SoC — exactly where fans and heat pipes live today in most designs. Apple has solved this architecturally with on-package unified memory. Every other OEM must solve it in the chassis.
Traditional fan-based cooling approaches will not readily work in the next generation of AI notebooks. Fans will either have to shrink (and get louder) or disappear entirely, giving way to other thermal-management solutions that allow for the necessary denser memory couplings surrounding the SoC. In this context, solid-state cooling is the most credible alternative on the table. With all this in mind, OEMs that treat thermal design as a fundamental design challenge rooted in geometry, rather than a late-stage design trade-off, set themselves up to lead in the emerging era of on-device inference.
The stakes for getting this right extend beyond consumer electronics. With GPU export controls creating real challenges, there is a rising global demand for local inference as an alternative to cloud computing. At this writing, the Mac Studio backlog through mid-2026 is the clearest signal of this. Premium laptops paired with distributed compute tools could become meaningful sovereign AI infrastructure, if the thermal and geometric problems get solved. Moor Insights & Strategy (MI&S) believes that the companies that resolve these constraints will define this burgeoning market.
Click the image below to download the full research brief:
[Image: Moor Insights & Strategy, generated with AI]
TABLE OF CONTENTS
Summary
Market Context and Bandwidth Requirements for AI
Physical Constraints: Geometry, Power, and Thermal Limits
Client Silicon Architecture Assessment
OEM Design Impact and Thermal Design Responses
Sovereign AI and Edge Infrastructure Considerations
Analyst Conclusions and Recommendations
COMPANIES CITED
Apple
Qualcomm
AMD
NVIDIA
Intel
HP
Samsung
SK Hynix
Micron
MediaTek
Microsoft
Ventiva
Frore
Cult of Mac
Synopsys
SamMobile
