High Bandwidth Memory (HBM): A Full Guide to How It Works, Who Makes It, and Why AI Depends on It

High-bandwidth memory is not a faster stick of RAM. It is DRAM stacked into a tower, wired through thousands of vertical connections, and placed a few millimeters from the processor so an AI chip can move terabytes of data every second.

In 2026 that stack became the gating item for the AI buildout. SK hynix, Samsung, and Micron are all in volume production of HBM4. NVIDIA's Vera Rubin platform is designed around it. And the reason accelerator lead times stay long is often not the logic die. It is the memory sitting next to it, and the packaging line that has to join the two.

Key takeaways

• HBM is stacked DRAM with a very wide bus. HBM4 doubles the interface to 2,048 bits and, at the JEDEC baseline of 8 Gb/s per pin, delivers up to 2 TB/s per stack.
• Vendor parts already exceed the standard. Samsung quotes a consistent 11.7 Gb/s (up to 13 Gb/s). Micron's 36 GB 12-high HBM4 is specified at more than 11 Gb/s and more than 2.8 TB/s. SK hynix says it is running above 10 Gb/s with more than 40% better power efficiency than the prior generation.
• Only three companies make it in volume. SK hynix, Samsung, and Micron account for essentially all commercial HBM. Share figures published by research firms should be treated as estimates; the vendors do not disclose HBM revenue as a line item.
• The bottleneck is packaging as much as silicon. An HBM stack is useless until it is attached to a GPU or ASIC on a 2.5D interposer. That attach step, not the DRAM wafer, is what most often limits shipments.
• HBM4 is a 2026 product, not a 2028 roadmap slide. Samsung began commercial shipments in February 2026. Micron entered high-volume production of 36 GB 12-high stacks in Q1 2026 for Vera Rubin. SK hynix completed development in September 2025 and ramped volume through the first half of 2026.
• It will not replace ordinary DRAM. HBM is expensive, thermally tight, and reserved for accelerators and a few high-end CPUs. Servers still run on DDR and, increasingly, low-power modules such as SOCAMM.

HBM at a glance

What HBM actually is

Start with the thing it is not. HBM will not speed up a laptop, and it is not the reason your phone got a few more gigabytes of RAM.

Ordinary DRAM, whether DDR5 in a server or LPDDR in a phone, sits centimeters away from the processor and talks over a relatively narrow channel. A typical DDR5 DIMM is 64 bits wide. That geometry is cheap and serviceable. It cannot feed a modern AI accelerator.

HBM's answer is geometric. Stack 4 to 16 DRAM dies on top of a logic base die. Drill the stack with thousands of through-silicon vias so the dies talk vertically. Then place the whole cube a few millimeters from the GPU or ASIC on a silicon interposer, and give it a bus that is 16 to 32 times wider than a DIMM.

The result is bandwidth measured in terabytes per second rather than hundreds of gigabytes, at a power cost per bit that conventional off-package DRAM cannot match. The price is complexity. Every extra die in the stack is another yield, warpage, and thermal problem, which is why only three memory companies ship HBM in volume and why advanced packaging has become as strategic as the DRAM process itself.

The problem HBM is solving

AI accelerators are memory-bandwidth machines wearing a compute costume. A matrix multiply only runs at advertised teraflops if the weights and activations arrive on time. When they do not, the chip stalls. That stall is why HBM exists, and why the industry will pay several times the cost of commodity DRAM to get it.

This is also why architectures that refuse HBM look so different. Groq's LPU keeps the working set in on-chip SRAM and accepts a tiny per-chip capacity as the cost of that bandwidth. Most of the industry took the other trade: large off-die capacity, attached as tightly as physics and packaging will allow.

How it works, layer by layer

A working HBM subsystem is four problems stacked on top of each other.

The DRAM stack

Each layer is a DRAM die, typically 24 Gb or 32 Gb in the HBM4 generation. JEDEC's JESD270-4 standard, published on 16 April 2025, allows 4-, 8-, 12-, and 16-high stacks. A 16-high stack of 32 Gb dies reaches 64 GB in a single cube.

Taller is not automatically better. Each added die increases the number of TSVs that must yield, the heat that must leave the stack, and the mechanical stress on the joints. That is why 12-high 36 GB parts are the 2026 volume workhorse, and why 16-high 48 GB and 64 GB cubes are still the harder, later product.

Through-silicon vias and the base die

TSVs are vertical copper connections drilled through the silicon. They are the reason a stack can behave like one very wide, very short memory channel instead of a pile of separate chips.

Under the DRAM sits a logic base die. It is not memory. It is the interface, the repair logic, the test access, and, increasingly, a place to put more of the PHY. Samsung's HBM4 base die is built on a 4 nm logic process, which is a different factory and a different yield problem from the DRAM above it. SK hynix has leaned on its Advanced MR-MUF process — mass reflow with molded underfill — to control warpage and heat as stacks get taller.

The interposer

The stack does not plug into a socket. It sits on a silicon interposer, or a silicon-bridge / organic equivalent, next to the processor. The interposer is a wiring board with lines fine enough to carry a 2,048-bit bus a few millimeters. TSMC's CoWoS family is the dominant version of this idea. Intel and Samsung have their own 2.5D flows.

This is the step that turns two good chips into one sellable accelerator, and it is the step that most often determines how many accelerators the industry can actually ship. We return to that below.

Power, heat, and refresh

HBM4's standard allows vendor-specific VDDQ levels from 0.7 V to 0.9 V and VDDC of 1.0 V or 1.05 V. Lower voltage is how the generation claims better picojoules per bit. It is also why power-efficiency numbers from vendors are not directly comparable: they depend on the workload mix, the stack height, and the calculator behind the press release.

Refresh and reliability matter more as stacks get denser. HBM4 adds directed refresh management, aimed at row-hammer mitigation and RAS. That is not a marketing feature. A 12-high cube sitting next to a 1,000-watt-class GPU is a hostile thermal environment for DRAM cells.

The generations, compared

The headline change in HBM4 is not a faster pin. It is a wider bus. JEDEC set the baseline pin rate at 8 Gb/s — actually below HBM3E's peak — and doubled the interface from 1,024 to 2,048 bits. Bandwidth still jumps because there are twice as many lanes.

Vendors then overclock the standard. That is why you will see 10, 11, or 13 Gb/s in product briefs next to a 8 Gb/s JEDEC number. Both can be true. One is the interoperable floor. The other is what a specific stack can do on a specific platform after qualification.

HBM4 is also backwards compatible with HBM3 controllers at the protocol level. That does not mean you can drop an HBM4 cube onto a Blackwell package. The physical attach, the thermal design, and the PHY are platform-specific.

What actually shipped in 2026

This was the year HBM4 left the sample lab and became a procurement item.

Two caveats belong next to that table.

First, "first" is a marketing word with several meanings. SK hynix was first to finish development. Samsung was first to announce commercial shipments. Micron was first to put a hard HBM4 revenue number on an earnings call. Those claims can coexist.

Second, qualification is not the same as allocation. NVIDIA saying all three vendors are in production means Vera Rubin is not hostage to one memory company. It does not mean a startup ASIC can buy the same cubes on similar terms.

Who makes it

The commercial HBM market is a three-firm industry. That concentration is not a temporary accident. The process combines leading-edge DRAM, a logic base die, TSV yield, and a stacking flow that took each vendor years to stabilize.

SK hynix

The incumbent. SK hynix defined the HBM3E cycle and entered HBM4 as the company everyone else was measured against. Its September 2025 announcement said HBM4 doubles bandwidth through 2,048 I/O terminals and improves power efficiency by more than 40%, with operating speed above 10 Gb/s against the 8 Gb/s JEDEC floor. It kept the Advanced MR-MUF stacking flow that it credits for warpage control and heat dissipation, and built the DRAM on its 1b-class (fifth-generation 10 nm-class) process to reduce mass-production risk.

Research-firm share estimates for SK hynix in HBM have typically sat in the mid-50s to low-60s percent range through 2025 and into 2026. Treat those as estimates. The company does not publish an HBM revenue line.

Samsung

The comeback. Samsung missed the early HBM3E NVIDIA socket and spent 2025 paying for that miss. On 12 February 2026 it announced that it had begun mass production of HBM4 and shipped commercial product, quoting 11.7 Gb/s consistent pin speed, headroom to 13 Gb/s, a 4 nm logic base die, and a 1c (sixth-generation 10 nm-class) DRAM process. It also said it expected HBM sales to more than triple in 2026 versus 2025, with HBM4E samples in the second half of 2026 and custom HBM samples in 2027.

The 4 nm base die is the distinctive technical bet. Putting more of the interface on a leading-edge logic process can improve speed and power. It also couples Samsung's HBM fate to its foundry, for better and worse.

Micron

The third source that became a real one. Micron's 16 March 2026 GTC announcement put 36 GB 12-high HBM4 in high-volume production for Vera Rubin, at more than 11 Gb/s and more than 2.8 TB/s, with a claimed 2.3× bandwidth and more than 20% power-efficiency gain versus its own HBM3E in the same 36 GB 12-high configuration. It had already shipped 48 GB 16-high samples. It also said 2026 HBM4 capacity was sold out under binding contracts.

Micron is the only one of the three that has, in 2026, put a hard HBM4 revenue figure on the public record (more than $1 billion shipped, disclosed on an earnings call). That single number is more useful than most share-percentage graphics.

Everyone else

There is no fourth volume supplier. Chinese memory companies have HBM programs, and several logic companies are experimenting with custom base dies or alternative stacking. None of that changes the 2026 procurement map. If you need HBM4 cubes this year, you are talking to one of the three.

Why packaging, not the DRAM wafer, is the real constraint

An HBM stack is a component. An AI accelerator is a package: one or more logic dies, several HBM cubes, an interposer, a substrate, and a thermal solution that has to survive a rack that now looks more like a power plant than a server row.

TSMC's CoWoS family is the main way those pieces become a product. Industry estimates for mid-2026 CoWoS capacity cluster around 120,000 to 140,000 wafers per month, with additional OSAT capacity from ASE, SPIL, and Powertech. Those figures are not TSMC guidance. They are reconstructed from supply-chain reporting, and they move. What does not move is the qualitative fact: CoWoS-class attach has been sold out through this cycle even as capacity roughly doubled.

That is why the state of AI infrastructure in 2026 is a story about power, cooling, and packaging together, not about GPU architecture in isolation. A Blackwell or Rubin GPU that cannot be paired with HBM on an interposer is unfinished inventory.

The same constraint is why AI chip startups challenging NVIDIA so often discover that the hard part is not drawing a better datapath. It is getting a CoWoS slot and an HBM allocation in the same quarter.

What it costs, and who can buy it

HBM is priced like a strategic input, not like commodity DRAM. Public stack-level prices are scarce and stale the moment they appear. The useful facts are structural.

• HBM sells at a large premium to DDR of the same bit count, because the TSV, base die, and stacking yield are expensive and because demand is concentrated in a handful of buyers.
• 2026 HBM4 capacity at the three vendors has been described as sold out or tightly allocated, especially for NVIDIA-qualified cubes. Micron said so explicitly for its own HBM4.
• The buyer list is short. NVIDIA, AMD, the hyperscaler ASIC programs (Google TPU, Amazon Trainium, Microsoft Maia, Meta MTIA), and a small number of other accelerator vendors. Everyone else buys residual.
• The cost that matters is not dollars per GB. It is dollars per terabyte-second of bandwidth, plus the opportunity cost of a CoWoS slot. That is the right unit when you compare HBM against SRAM-heavy designs or against inference-chip total cost of ownership.

If a vendor quotes you an HBM price without a stack height, a pin speed, a qualification status, and a packaging path, they are not quoting you a real part.

What is real and what is hype

Real

• HBM4 is in commercial production at all three suppliers in 2026, and it is designed into the next NVIDIA platform.
• The architectural jump is the 2,048-bit interface, not a heroic pin-speed increase.
• Packaging attach, not DRAM wafer starts, is the binding constraint on how many HBM-equipped accelerators the industry can ship.
• Power per bit moved is a first-order data-center issue. Vendor efficiency claims of 20% to 40% versus HBM3E are directionally consistent even if the exact percentages are not comparable across companies.

Hype

• "HBM solves the memory wall." It moves the wall. Context length, expert-MoE routing, and multi-host inference still run out of capacity and bandwidth. HBM4's 36–64 GB per stack is large until you put eight stacks on a GPU and then try to hold a frontier model plus KV cache.
• "A fourth supplier is about to break the oligopoly." Not in 2026, and not with NVIDIA-qualified HBM4.
• "HBM4E / custom HBM ends the shortage." Those products are samples and 2027 conversations. They do not loosen 2026 allocation.
• "Any AI chip can just add HBM." Without a 2.5D packaging slot, a qualified PHY, and a thermal design that can cool a 12-high cube next to a hot logic die, you have a slide, not a product.

A realistic timeline

One useful heuristic: watch 16-high yield and CoWoS-L / CoPoS attach, not press-release pin speeds. The generation that ships in volume at acceptable yield is the generation that matters.

The bottom line

2026 is the year HBM stopped being a component that GPU companies mentioned in a footnote and became the thing that determines who gets to ship.

The physics is settled. Stacked DRAM with a 2,048-bit bus is how you feed an accelerator that burns a kilowatt. The JEDEC floor is 2 TB/s per stack. The parts actually shipping are faster than that. The industrial problem is yielding those stacks, attaching them, and allocating them to a buyer list that is shorter than the demand list.

For anyone making decisions today, the split is clean. If the question is how to buy AI compute this year, HBM allocation and packaging slots are part of the bill of materials, not a later integration detail. If the question is whether a new accelerator architecture is real, ask where the weights live, how many terabytes per second they can move, and who has promised the cubes.

Frequently asked questions

What is high-bandwidth memory (HBM)?

High-bandwidth memory is DRAM stacked vertically into a cube and placed on the same package as a processor, connected through a bus that is 1,024 or 2,048 bits wide. The point is to move data at terabytes per second over millimeters of interconnect instead of hundreds of gigabytes per second over a DIMM channel. It is the standard memory for AI GPUs and many custom accelerators.

How is HBM different from GDDR or DDR5?

DDR5 and GDDR sit off-package and use much narrower interfaces. HBM sits on-package, uses through-silicon vias to stack dies, and pays for that density with a harder manufacturing flow and a much higher price. GDDR is still the right answer for most discrete PC graphics cards. DDR5 is still the right answer for host memory. HBM is the right answer when an accelerator would otherwise stall on memory bandwidth.

What is new in HBM4?

HBM4, standardized by JEDEC as JESD270-4 in April 2025, doubles the interface to 2,048 bits and the channel count to 32, supports stacks up to 16-high and 64 GB, and sets a baseline of 8 Gb/s per pin for 2 TB/s per stack. Vendor products in 2026 run faster than that baseline. The generation is also designed to be backwards compatible with HBM3 controllers at the protocol level.

Who makes HBM?

SK hynix, Samsung Electronics, and Micron Technology. Together they produce essentially all commercial HBM. SK hynix led the HBM3E cycle. Samsung was first to announce commercial HBM4 shipments in February 2026. Micron put 36 GB 12-high HBM4 into high-volume production for NVIDIA Vera Rubin in the first quarter of 2026.

Why is HBM in short supply?

Because demand is concentrated in a few AI platforms, because only three companies can yield the stacks, and because each stack still has to be attached to a logic die on a scarce 2.5D packaging line. Adding DRAM wafer capacity does not help if the stacking flow or the CoWoS slot is the limiting step.

How much HBM does an AI GPU use?

It depends on the platform. A modern high-end accelerator typically carries multiple stacks — often six to eight — for total capacities in the 192–288 GB range on current HBM3E parts, with HBM4 platforms moving higher as 12-high and then 16-high cubes qualify. Always check the specific SKU. "The GPU has HBM" is not a capacity number.

Is HBM only for NVIDIA?

No. AMD, Google, Amazon, Microsoft, Meta, and other accelerator designers use HBM or equivalent on-package stacked memory. NVIDIA is the largest single buyer, which is why its qualification decisions move the entire market, but it is not the only customer.

Will HBM be replaced by SRAM or optical memory?

Not as a general solution. On-chip SRAM, as in Groq's LPU, wins latency and bandwidth at the cost of capacity. Optical or photonics-ready memory is a research and early-product story, not a 2026 procurement option. The industry's actual plan is more HBM, taller stacks, custom base dies, and better packaging — plus, for some inference workloads, architectures that need less memory movement in the first place.

Methodology

This guide is based on primary company and standards disclosures available as of 16 August 2026: JEDEC's JESD270-4 HBM4 announcement (16 April 2025), SK hynix's 12 September 2025 HBM4 development release, Samsung's 12 February 2026 commercial-shipment release, and Micron's 16 March 2026 high-volume-production release. Market-share and CoWoS-capacity figures are labeled as research-firm or supply-chain estimates, not as filings. Vendor power-efficiency percentages are company-stated and are not normalized to a common workload.