LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

DDR5 and Why Memory Latency Stopped Improving

12 min read The Physics of Computing
ddr5memorydramhardwareperformancelatency
Contents

Over the last twenty years, the capacity of a memory module grew by roughly 128 times and its bandwidth by about 20 times. In that same span, the one number that determines how long your CPU waits for a byte it did not see coming — the latency to fetch a random word out of DRAM — improved by a factor of about 1.3. Not 1.3 times per generation; 1.3 times total, across two decades and three full generations of the standard. A random read from main memory took something like 13 to 15 nanoseconds in the DDR3 era, and it takes something like 13 to 15 nanoseconds today on DDR5, which is why a modern 5 GHz core that can retire several instructions per cycle will sit and stall for the equivalent of hundreds of instructions every time it misses cache and has to go to RAM. DDR5 is a genuinely large step forward — but every headline improvement it brought is about bandwidth and reliability, and almost none of it is about that stubborn first-byte latency, because the thing setting that latency is not the interface DDR5 redesigned. It is the analog physics of the memory cell, and the physics has not moved.

Why a DRAM Read Is Slow in the First Place

A DRAM bit is the simplest possible storage element: one transistor and one capacitor, a design you can trace back through the transistor fundamentals. The capacitor holds a tiny charge that means 1 or 0; the transistor is the gate that connects it to a bit line when you want to read or write. This is brilliant for density — you can pack billions of these onto a die because each cell is microscopic — and it is the root of the latency problem, because reading that cell is an analog operation that takes a fixed amount of real-world time no matter how clever the interface around it gets.

When you read a DRAM location, a chain of physical events has to happen in order. The row’s word line is driven, connecting a whole row of capacitors to their bit lines. Each tiny capacitor dumps or withholds its minuscule charge, nudging the bit-line voltage by a hair. A sense amplifier then has to detect that nudge against noise and amplify it to a full logic level — and sensing a few femtocoulombs of charge takes time that is set by capacitance and analog settling, not by clock frequency. Only after the row is “open” in the sense amplifiers can the column be selected and the data driven out. The critical timings — tRCD (the delay from activating a row to being able to read a column) and CAS latency (the delay from the column command to data appearing) — are dominated by this analog sense-and-settle process. You can double the data rate of the bus all you like; the capacitor still discharges at the speed a capacitor discharges. That is the wall.

The Latency Number That Refuses to Move

The cruel arithmetic is visible right in the spec sheets, and it confuses nearly everyone. DDR5’s CAS latency numbers, measured in clock cycles, are higher than DDR4’s — DDR5-4800 ships at CL40 where DDR4-3200 was CL16 — and people read that and conclude DDR5 is slower. The cycle count is the wrong unit. What matters is the absolute time, which is the cycle count multiplied by how long a cycle lasts, and DDR5’s cycles are much shorter:

Memory Data rate Clock CAS (cycles) First-word latency
DDR3-1600 1600 MT/s 800 MHz CL11 ~13.75 ns
DDR4-3200 3200 MT/s 1600 MHz CL16 ~10.0 ns
DDR4-3200 3200 MT/s 1600 MHz CL22 ~13.75 ns
DDR5-4800 4800 MT/s 2400 MHz CL40 ~16.7 ns
DDR5-6400 6400 MT/s 3200 MHz CL36 ~11.25 ns
DDR5-8000 8000 MT/s 4000 MHz CL40 ~10.0 ns

Stare at the right-hand column. The first-word latency wanders between roughly 10 and 17 nanoseconds and goes essentially nowhere across three generations. Entry-level DDR5 is actually slower to first byte than good DDR4, and it takes high-clocked, tight-timing DDR5 just to tie DDR4. The latency to that first random byte has been stuck near 13 nanoseconds for fifteen years, because it is gated by the sense amplifier, and the sense amplifier does not care what generation is printed on the module.

What DDR5 Actually Changed: Bandwidth and Banks

If DDR5 cannot fix latency, what did it fix? Throughput and concurrency — and it did so substantially. The architecture is genuinely different from DDR4 in ways that matter for any workload that touches a lot of memory.

Two independent sub-channels per DIMM. A DDR4 module is one 64-bit-wide channel. A DDR5 module splits the same module into two independent 32-bit sub-channels (40 bits each counting ECC), and the memory controller can drive them in parallel with separate commands. Two narrower channels with independent command streams extract far more usable bandwidth from the same pins, because more requests are in flight and there is less waiting on a single shared command bus.

Double the bank groups. DDR5 doubles bank groups from 4 to 8 and total banks from 16 to 32. More banks means more rows can be open simultaneously, which matters enormously for random access: if the row you need is already open in some bank, you skip the activate step entirely. More banks statistically means more hits on already-open rows, which lowers effective latency for real, scattered access patterns even though the worst-case single-cell latency is unchanged. This is the one place DDR5 nibbles at latency, and it does so by hiding it through parallelism, not by making the cell faster.

Longer bursts. DDR5 moves to a burst length of 16 (BL16) against DDR4’s BL8, so a single access streams 64 bytes — exactly one CPU cache line — from one sub-channel. The whole interface is tuned to feed cache lines efficiently, which is what a CPU actually consumes.

   DDR4 DIMM                          DDR5 DIMM
  ┌──────────────────────┐          ┌───────────────────────────────┐
  │  one 64-bit channel  │          │ sub-ch A (32b) │ sub-ch B(32b) │
  │  16 banks / 4 groups │          │  independent commands, in      │
  │  command bus shared   │          │  parallel; 32 banks / 8 groups│
  └──────────┬───────────┘          └───────┬──────────────┬────────┘
             │ one request stream            │ two request streams
             v                               v              v
        more waiting on               more requests in flight,
        a single command bus          more open rows = fewer activates
        ──────────────────────────────────────────────────────────────
        BL8  = 64B over 8 beats        BL16 = 64B per sub-channel
        latency to first byte: ~13ns   latency to first byte: ~13ns  (unchanged)

The pattern is unmistakable: every DDR5 win is about getting more bytes per second and more requests overlapping, never about getting the first byte sooner. This is the same lesson the storage hierarchy keeps teaching — that the channel, not the cell, increasingly bounds memory — and it is exactly why the industry went looking for capacity and bandwidth elsewhere, in CXL’s switchable memory fabric and in the broader question of what comes after the current memory hierarchy.

On-Die ECC and the PMIC: Reliability Moves Onto the Module

Two more DDR5 changes are not about speed at all, and they reveal what was actually getting hard as cells shrank.

On-die ECC. DDR5 chips include error-correcting code on the die itself, correcting single-bit errors internally before data ever leaves the chip. This is not the same as the system-level ECC that server admins know — it does not report to the OS or catch errors on the bus — and it is not there as a feature so much as a necessity. At DDR5 cell densities, the bit cells are so small and hold so little charge that single-bit errors became frequent enough that the chips could not meet reliability targets without scrubbing them silently on-die. On-die ECC is the tax DRAM pays to keep shrinking; it is the cell physics pushing back, just like the latency wall, and the engineers answering it with redundancy instead of speed.

The PMIC. DDR4 modules took regulated voltage from the motherboard. DDR5 moves the Power Management IC onto the module: the board supplies a clean 5V (later 12V on some server designs), and an on-DIMM PMIC steps it down to the ~1.1V the chips need, right next to where it is consumed. Putting regulation millimeters from the load instead of across the board cuts noise and voltage droop, which is what lets the interface clock as high as it does. As with every multi-gigabit link, the signal-integrity reality sets the ceiling, and moving power local to the module is part of how DDR5 keeps the eye open at 6400 MT/s and beyond.

DDR5 change What it improves What it does not improve
Dual 32-bit sub-channels Usable bandwidth, request concurrency First-word latency
8 bank groups / 32 banks Effective latency via more open rows Worst-case cell latency
Burst length 16 Cache-line-efficient transfers Time to first byte
On-die ECC Reliability at shrunken cell sizes Performance (slight overhead, if any)
On-module PMIC Signal integrity, higher clocks Latency

How Systems Hide a Wall They Cannot Break

If the cell latency is fixed, why do modern machines feel faster every generation? Because the entire CPU has been organized for fifteen years around the assumption that main memory is slow and getting no faster, so it spends enormous transistor budgets hiding the wait rather than shortening it. The multi-level cache hierarchy exists precisely so most accesses never reach DRAM at all — an L1 hit is a nanosecond or two, an L3 hit maybe ten, and only a true miss pays the full ~80–100 ns round trip. Hardware prefetchers watch access patterns and fetch the next cache line before the core asks for it, converting a future stall into a background transfer. Out-of-order execution lets the core keep running dozens of independent instructions while one of them waits on memory, so the latency overlaps with useful work instead of stalling the pipeline. And when even that is not enough, designers throw parallelism at the problem: more memory channels, more banks, and exotic packaging like HBM that stacks DRAM dies and wires them with thousands of connections to multiply bandwidth — never latency. Every one of these is an admission that the first-byte latency is a fixed cost of doing business, the same architectural resignation that pushed capacity and bandwidth growth onto interconnects rather than cells.

What This Means When You Buy or Tune Memory

The practical consequences fall out of the physics directly. For latency-bound workloads — many games, single-threaded code, anything that chases pointers through scattered memory and stalls on each miss — raw DDR5 data rate buys you little, and what matters is keeping the effective latency low: a higher data rate paired with tight timings so the absolute CAS time stays near 10 ns, not a huge MT/s number with loose CL. For bandwidth-bound workloads — content creation, scientific computing, integrated graphics that share system memory, and anything feeding many cores — DDR5’s dual sub-channels and higher data rates are a real, large win, frequently 50% or more usable bandwidth over DDR4. Most server and HPC workloads, and the Linux memory subsystem’s appetite for streaming large datasets, fall on the bandwidth side and benefit straightforwardly.

A few commands make the reality visible on a Linux box:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# Module details: speed, ranks, manufacturer, configured vs max clock
sudo dmidecode -t memory | grep -E "Speed|Rank|Type:|Configured"

# Confirm DDR5 and the negotiated data rate
sudo lshw -short -C memory

# Measure what actually matters: latency and bandwidth, not the label
# (Intel Memory Latency Checker reports idle latency in ns + bandwidth)
sudo ./mlc --latency_matrix
sudo ./mlc --bandwidth_matrix

# Quick streaming-bandwidth sanity check
sysbench memory --memory-block-size=1M --memory-total-size=20G run

The number to anchor on is the idle latency mlc reports: it will land somewhere around 70–100 ns for a full random access including the controller and interconnect overhead, of which the DRAM cell itself is that stubborn ~13 ns, and no amount of MT/s on the box will move the cell’s contribution.

Verdict

DDR5 is a major generational advance that does almost nothing for the problem most people assume memory generations are about. The time to fetch the first random byte from DRAM has been pinned near 13 nanoseconds since the DDR3 era because it is set by an analog process — a sense amplifier detecting a few femtocoulombs of charge leaking off a microscopic capacitor — and that process is indifferent to how fast the bus around it runs. What DDR5 actually delivers is throughput and reliability: two independent sub-channels per module and twice the bank groups for far more bandwidth and request concurrency, longer bursts tuned to the cache line, on-die ECC that exists because cells got too small to be reliable without it, and an on-module PMIC that cleans up power so the interface can clock higher. Read the spec sheet in absolute nanoseconds, not cycle counts, and the truth is plain: entry DDR5 is slower to first byte than good DDR4, and it takes high-clocked tight-timing DDR5 merely to tie it. For buyers that means matching the memory to the workload — tight effective latency for pointer-chasing and games, raw bandwidth for everything that streams — and for the industry it means the latency wall is permanent enough that the interesting work has moved elsewhere: hiding latency behind parallelism, caching ever more aggressively, and building entirely new tiers like CXL to attack capacity and bandwidth, because the one thing no DDR generation can sell you is a faster capacitor.

Sources

Comments