Samsung and SK Hynix answer the memory bottleneck with vertical stacking and high-bandwidth flash
Both memory makers unveiled architectural work aimed squarely at AI's bandwidth constraint — "zHBM" vertical stacking and a new high-bandwidth flash standard. Compute has not been the binding limit on inference for some time; feeding it has.
The economics of serving a large model are dominated by moving weights, not multiplying them. Accelerators spend a large fraction of their cycles waiting on memory, which is why HBM supply — not GPU dies — has been the scarce input rationing the buildout.
zHBM's vertical stacking attacks capacity and bandwidth per package. High-bandwidth flash is the more interesting move: it proposes a tier between DRAM and storage for weights that are needed often enough to hurt when they are far away but not often enough to justify HBM's cost per bit. Mixture-of-experts models, where most parameters are idle on any given token, are exactly the workload that shape was designed for.
Standards work is slow and the announcements are architectural rather than shipping. But the direction is a useful corrective to a market that still prices AI capacity in GPUs.
Distill Intelligence — Semiconductors & AI Chips Weekly Briefing — August 7, 2026 →