LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

The Flash Translation Layer

13 min read The Physics of Computing
ftlssdnandgarbage-collectionwrite-amplificationstorage
Contents

Your SSD is lying to you, and the lie is the product. The operating system thinks it is talking to a disk — a flat array of fixed-size sectors you can overwrite in place, any one, any time, forever. NAND flash can do none of those things. You cannot overwrite a flash page; you must erase a much larger block first. You cannot erase a single page; erase only works on whole blocks of hundreds of pages at once. And every erase wears the cells out, so you cannot keep hammering the same location. The gap between the tidy disk the OS expects and the unruly flash that actually exists is bridged by a single, enormous piece of firmware running on the drive’s controller: the Flash Translation Layer. The FTL’s entire job is to maintain the fiction of a disk on top of a medium that behaves nothing like one, and how well it tells that lie determines whether your SSD is fast and durable or slow and dead.

Everything people find confusing about SSDs — why they slow down after they fill up, why writing 1 GB can wear out 3 GB of flash, why a brand-new drive benchmarks twice as fast as the same drive a week later, why TRIM matters, why the spare capacity you cannot see is the most important capacity on the device — is downstream of the FTL doing its job. This is the firmware that turns a hostile physical medium into the storage abstraction the entire computing stack is built on, and once you understand its constraints, the SSD’s whole personality becomes predictable instead of mysterious.

The impedance mismatch the FTL has to hide

To understand the FTL you have to first respect exactly how badly flash mismatches the disk abstraction. A hard drive (how data lives on platters and flash) lets you rewrite any 512-byte sector in place as many times as you like. NAND flash imposes three hard rules that break all of that:

  • Read and program (write) happen at page granularity — typically 4 KB to 16 KB. Fine.
  • Erase happens only at block granularity — a block is hundreds of pages, often a few megabytes. You cannot erase less.
  • You must erase before you can re-program. A page holds data until its entire block is erased; you cannot overwrite a page in place. Programming only flips bits one direction; erase resets the whole block.
  • Erase wears cells out. Each block tolerates a finite number of program/erase (P/E) cycles before its read window collapses and it can no longer hold data reliably.

Put those together and the consequence is brutal: there is no such thing as “modify a sector” in flash. If the host wants to change 4 KB that currently lives in the middle of a full block, you cannot just rewrite those 4 KB. You would have to read the whole block, erase it, and rewrite everything — for a 4 KB change. No drive does that. Instead the FTL does the only sane thing: it writes the new data to a fresh, already-erased page somewhere else, and updates a map so that the old logical address now points to the new physical location. The old page is simply marked stale and dealt with later. This indirection — host addresses decoupled from physical locations — is the foundational trick, and it is why the FTL needs a map.

L2P: the map that is the FTL’s beating heart

The core data structure of every FTL is the logical-to-physical (L2P) mapping table. The host addresses storage by logical block address (LBA); the flash stores data at physical page addresses; the L2P table translates one to the other on every single access. When you write LBA 12345, the FTL picks a free physical page, programs the data there, and records “LBA 12345 -> physical page N.” When you later read LBA 12345, it consults the map to find page N. The disk the OS sees is entirely a projection of this table.

The granularity of that map is a defining design choice with real cost tradeoffs:

Mapping scheme Granularity RAM needed Write amplification Used by
Page-level One entry per page (~4 KB) High (~1 GB DRAM per 1 TB) Low Mainstream DRAM SSDs
Block-level One entry per block (~MB) Low High (whole-block rewrites) Obsolete / tiny devices
Hybrid (log-block) Block map + small page-mapped log Medium Medium DRAM-less, eMMC, SD cards

Page-level mapping is the gold standard because it gives the FTL maximum freedom to place any page anywhere, which minimizes wasted work — but it is expensive in RAM. The rule of thumb is 4 bytes of map per 4 KB of flash, which works out to roughly 1 GB of DRAM per 1 TB of capacity, and that DRAM is why mainstream SSDs have a DRAM chip on board. DRAM-less drives cut that cost by keeping only a fraction of the map resident — using the NVMe Host Memory Buffer (HMB) to borrow a little system RAM, or a hybrid scheme — and they pay for it with more internal data movement and worse random-write behavior. There is no free lunch: you pay for the map in DRAM, or you pay for its absence in performance.

   Host view (the lie)            FTL reality (the truth)
   ┌───────────────────┐         ┌──────────────────────────┐
   │ LBA 0             │         │  L2P table (in DRAM)      │
   │ LBA 1   ──────────┼────────>│  LBA 12345 -> PPA 0x4F210 │
   │ ...               │         │  LBA 12346 -> PPA 0x91A08 │
   │ LBA 12345         │         └────────────┬─────────────┘
   │ ...               │                      │ resolves to
   │ "a flat disk"     │                      ▼
   └───────────────────┘         ┌──────────────────────────┐
                                 │ NAND: blocks of pages,    │
                                 │ erase-before-write, wear  │
                                 └──────────────────────────┘

The map itself must survive power loss, so the FTL journals it to flash and rebuilds it on boot — a whole sub-discipline of its own that overlaps with power-loss protection. A corrupted L2P table is a bricked drive, because without the map the physical pages are just anonymous bytes with no addresses.

Garbage collection and the write-amplification tax

Because every overwrite writes to a fresh page and abandons the old one, a working SSD steadily fills with stale pages — data that has been superseded but still occupies physical flash. Those pages are trapped: you cannot reclaim them individually, because erase works only on whole blocks, and the blocks holding stale pages also hold valid pages mixed in. Reclaiming space is the job of garbage collection (GC), and GC is where the FTL’s hardest tradeoffs live.

GC picks a victim block, copies its still-valid pages out to a fresh block, and then erases the victim to return it to the free pool. The problem is that copy step. To free a block that is, say, 75% valid, the FTL must first write that 75% somewhere else — extra flash writes the host never asked for. This is write amplification, and it is measured by the Write Amplification Factor:

   WAF = (data written to NAND) / (data written by host)

   Host writes 1 GB, drive programs 3 GB of NAND  ->  WAF = 3.0

   Ideal (no GC overhead):           WAF = 1.0
   Random writes, full drive:        WAF can exceed 3-5
   Sequential writes:                WAF approaches 1.0

WAF matters for two reasons, and they compound. First, performance: every amplified write competes with host writes for the flash channels, so a high WAF directly throttles throughput. Second, endurance: the flash has a finite P/E budget, so a WAF of 3 burns your drive’s lifespan three times faster than the host workload implies. A drive rated for 600 TBW (terabytes written by the host) at WAF 1 might deliver only 200 TBW of useful host writes if its real-world WAF is 3. This is the same endurance ledger that NAND reliability and qual accounts for at the cell level — the FTL is the thing spending that budget.

Victim selection is its own art. The simplest policy, greedy GC, always picks the block with the fewest valid pages, minimizing immediate copy cost. Smarter cost-benefit policies also weigh how long data has been stable, preferring to leave hot, frequently-rewritten data alone (it will invalidate itself soon anyway) and to compact cold, stable data that is just sitting there. Separating hot from cold data into different blocks — data placement — is one of the highest-leverage things an FTL can do, because mixing a hot page into a block of cold pages means GC keeps dragging that whole block around.

Overprovisioning: the master knob

Here is the single most important lever over write amplification, and it is invisible to the user: overprovisioning (OP), the physical flash capacity the drive keeps for itself and never exposes to the host.

   % OP = 100 * (physical_flash_capacity - exposed_capacity) / exposed_capacity

   Example: 512 GiB of flash, advertised as 480 GB
            roughly 7% baseline OP (plus the GiB/GB ~7.37% gap)
   Enterprise drives: deliberately advertise 3.84 TB on 4 TiB
            -> ~28% OP for sustained write performance

Why does hidden capacity make writes faster and the drive last longer? Because GC efficiency depends on how empty the average victim block is, and OP guarantees a permanent reserve of free space that the host can never consume. The more spare room the FTL has, the more likely a victim block is mostly-stale when collected, so less valid data has to be copied per block freed — directly lowering WAF. The relationship is steep and nonlinear: at very low spare space WAF climbs toward a cliff, while adding OP buys rapidly diminishing but real reductions.

  WAF
   6 ┤●
     │ \
   4 ┤  ●            little spare -> brutal amplification
     │   \●
   3 ┤     ●
     │       ●●____  7% OP (consumer)
   2 ┤            ●●●____
     │                  ●●●●____  28% OP (enterprise)
   1 ┤- - - - - - - - - - - - - ●●●●●●●  ideal
     └──┬─────┬─────┬─────┬─────┬───> overprovisioning %
        0     7     14    21    28

This is why enterprise SSDs with identical NAND to consumer drives are sold at lower advertised capacities and command a premium: you are paying for the withheld flash, which the controller spends on sustained, low-WAF write performance. And it is a knob you can turn yourself — leaving a drive’s last 10-20% unpartitioned (or short-stroking it) gives the FTL more effective spare and measurably improves steady-state write performance and endurance. The capacity you do not use is working for you.

Wear leveling and TRIM

Two more FTL responsibilities round out the picture, and both are about long-term health rather than moment-to-moment translation.

Wear leveling ensures the P/E cycles spread evenly across all blocks, because a drive dies when its first blocks wear out, not its average ones. Dynamic wear leveling naturally rotates writes through the pool of free blocks, but it only touches blocks that are actively being rewritten. The subtle failure it misses is static data: a file written once and never touched sits in a block that accrues zero wear, while the free pool churns and ages around it. Static wear leveling fixes this by periodically and deliberately relocating cold, stable data to free up its low-wear blocks for rotation — spending a little write amplification now to avoid a lopsided early death later. It is the FTL choosing to do extra work to extend the drive’s life, and getting the balance right is genuinely hard.

TRIM closes an information gap that otherwise cripples GC. When you delete a file, the filesystem marks those LBAs free — but the SSD has no idea, because at the block layer a delete is silent. Without help, the FTL still believes that old data is valid and dutifully copies it during GC, wasting writes preserving bytes nobody wants. The TRIM command (NVMe Dataset Management Deallocate, SCSI UNMAP) lets the OS tell the drive “these LBAs are now garbage,” so the FTL can drop them from the map and skip relocating them — instantly lowering valid-data counts and WAF. (Note this is a completely different “trim” from the analog cell-calibration trim settings configured at the factory; the collision of terms is an industry hazard.) A filesystem like ZFS that understands the underlying device (ZFS for the homelab) issues TRIM intelligently, and the difference in sustained performance on a full drive is dramatic.

Steady state: why fresh-out-of-box benchmarks lie

Now the payoff that ties it all together: the reason you cannot trust an SSD benchmark unless it was taken in steady state. A fresh drive, or one that has just been secure-erased, is in a fool’s-paradise condition called fresh-out-of-box (FOB): every block is erased and free, the L2P map is nearly empty, and there is no stale data anywhere. In that state, GC has nothing to do, WAF is effectively 1.0, and every host write goes straight to a waiting free page. The drive posts spectacular numbers.

That state does not last. As you write, free blocks get consumed, stale pages accumulate, and eventually the drive runs out of pre-erased space and must garbage-collect in the write path — freeing blocks on demand to service incoming writes. The moment that happens, performance falls off the write cliff: throughput drops, sometimes by half or more, and stabilizes at the true sustained rate the drive can hold indefinitely. Many drives also paper over the early period with an SLC-mode cache (the pSLC apology), which posts even higher burst numbers that collapse once the cache fills and has to drain.

 throughput
   high ┤●●●●●●            FOB / SLC-cache burst (marketing)
        │       ●●
        │         ●         <-- the write cliff
        │          ●●●
    low ┤             ●●●●●●●●●●●●●●  true steady state (reality)
        └────────────────────────────> data written
              cache+free      GC-in-path

This is exactly why the storage industry standardized honest measurement. The SNIA Solid State Storage Performance Test Specification requires preconditioning a drive — writing it well past full and into stable behavior — before recording any number, so that the published figure reflects sustained reality rather than FOB fantasy. The same discipline matters for any benchmark of any SSD subsystem, including the ones discussed in how semiconductors are tested: a measurement taken before the system reaches its operating equilibrium is not a measurement, it is an advertisement. When you evaluate a drive, the only number that means anything is the one it sustains after the cliff.

Verdict

The Flash Translation Layer is the most consequential firmware most engineers never think about: it is the lie that makes a hostile, erase-before-write, wear-limited medium impersonate a friendly disk, and the quality of that lie is the quality of the SSD. Every confusing SSD behavior decodes from its constraints. The L2P map is the heart of the fiction and costs you roughly 1 GB of DRAM per 1 TB, which is why DRAM-less drives trade that RAM for worse random writes. Garbage collection is unavoidable because erase is block-granular, and it taxes you through write amplification that burns both performance and endurance — a WAF of 3 means a third of your write budget is the FTL’s own bookkeeping. Overprovisioning is the master knob that buys down WAF by guaranteeing spare room, which is why enterprise drives are just consumer NAND sold at a lower advertised capacity, and why leaving part of your own drive unpartitioned genuinely helps. Wear leveling keeps the first block from dying early, and TRIM keeps GC from preserving data you already deleted. And the whole edifice explains the most important practical lesson in storage benchmarking: a fresh drive’s numbers are fiction, the write cliff is real, and steady state measured after SNIA-style preconditioning is the only honest figure. Understand the FTL and the SSD stops being a black box that mysteriously slows down, and becomes a predictable machine doing exactly the work its physics demands.

Sources

Comments