Linux Memory Management Deep Dive

Memory management is one of those kernel subsystems that most engineers interact with only when something goes wrong — an OOM kill, a mysterious slowdown, a latency spike that correlates with nothing obvious. Understanding how Linux manages memory doesn’t just help you debug these situations; it changes how you architect systems in the first place.

This post covers the full stack: virtual memory fundamentals, the page allocator, huge pages (transparent and explicit), NUMA topology, the OOM killer, and the tools for measuring and tuning memory behavior.

Virtual Memory Fundamentals

Every process on Linux operates in a private virtual address space. The kernel maintains a mapping from virtual addresses to physical frames — the page table. This separation gives the OS several important properties:

• Isolation: processes can’t access each other’s memory (unless explicitly shared)
• Overcommit: the virtual address space can exceed physical RAM
• Lazy allocation: memory is only backed by physical pages on first access

The Page Table Walk

On x86-64, the page table is a 4-level (or 5-level on recent CPUs) radix tree:

Virtual Address (48-bit):
  [PML4 index][PDP index][PD index][PT index][Page offset]
  [  9 bits  ][ 9 bits  ][ 9 bits ][ 9 bits ][  12 bits  ]

Each level is a table of 512 64-bit entries, each pointing to the next level or (at the leaf) to a physical page frame. A full walk requires 4 memory accesses for the table entries plus 1 for the actual data — 5 total. The Translation Lookaside Buffer (TLB) caches recent translations to amortize this cost.

TLB misses are expensive. On modern CPUs, a TLB miss can take 100–300 cycles to service if the page tables are cold in cache. This is why huge pages matter.

The Virtual Memory Areas (VMAs)

The kernel tracks a process’s virtual address space as a sorted list (and rbtree) of vm_area_struct objects, each describing a contiguous region with uniform permissions and backing:

1
2

# View a process's VMAs
cat /proc/$(pgrep nginx | head -1)/maps

Output:

55a1b4a00000-55a1b4b2e000 r-xp 00000000 08:01 1234567 /usr/sbin/nginx   # text
55a1b4d2e000-55a1b4d3b000 r--p 0012e000 08:01 1234567 /usr/sbin/nginx   # rodata
55a1b4d3b000-55a1b4d3e000 rw-p 0013b000 08:01 1234567 /usr/sbin/nginx   # data
55a1b5000000-55a1b5200000 rw-p 00000000 00:00 0        [heap]
7f8a00000000-7f8a40000000 rw-p 00000000 00:00 0        [anonymous]
7ffd12345000-7ffd12366000 rw-p 00000000 00:00 0        [stack]

/proc/PID/smaps gives per-VMA statistics including RSS (resident), PSS (proportional share), and swap usage.

Page Allocation

The Buddy Allocator

The kernel’s physical memory allocator is the buddy system. Physical memory is organized into zones (DMA, Normal, highmem on 32-bit), and within each zone, free pages are tracked in 11 free lists, one for each order from 2^0 (4 KB) to 2^10 (4 MB).

To allocate a page of order N:

1. Look in the order-N free list
2. If empty, split a block from order N+1, put one half back, return the other
3. On free, check if the “buddy” is also free; if so, merge back up

This gives O(log n) allocation and keeps fragmentation bounded. But over time, memory fragments — it becomes hard to find contiguous ranges for large allocations.

1
2
3
4
5

# View buddy allocator state
cat /proc/buddyinfo
# Node 0, zone   Normal  4234 3012 1589 801 423 198 88 42 19 8 2
# Orders:         0       1    2    3   4   5  6  7  8  9  10
# Higher order = larger free contiguous blocks

The Slab Allocator

The buddy allocator works at page granularity (4 KB). For kernel objects that are much smaller (inodes, dentries, socket buffers), the slab allocator manages per-object caches on top of pages from the buddy allocator.

1
2
3
4

# View slab cache usage
cat /proc/slabinfo | head -20
# or
sudo slabtop

The slab allocator (SLUB in modern kernels) dramatically reduces internal fragmentation for small kernel objects and improves cache locality by keeping same-type objects together.

/proc/meminfo Explained

/proc/meminfo is the primary window into kernel memory state. The fields that actually matter:

1

cat /proc/meminfo

MemTotal:       131072000 kB   # Total usable RAM
MemFree:          512000 kB   # Completely idle pages
MemAvailable:   98304000 kB   # Estimated available for new processes (*)
Buffers:         2048000 kB   # Page cache for block device metadata
Cached:         24576000 kB   # Page cache for file data
SwapCached:           0 kB   # Pages in swap that are also in RAM (read back but not evicted yet)
Active:         45000000 kB   # Recently used, unlikely to be reclaimed
Inactive:       28000000 kB   # Not recently used, candidate for reclaim
Active(anon):   20000000 kB   # Anonymous active (heap, stack, mmapped files)
Inactive(anon):  8000000 kB   # Anonymous inactive
Active(file):   25000000 kB   # File-backed active (page cache)
Inactive(file): 20000000 kB   # File-backed inactive
Unevictable:     1024000 kB   # Locked pages (mlock), cannot be reclaimed
Mlocked:          512000 kB   # Pages locked with mlock()
SwapTotal:      16777216 kB   # Total swap
SwapFree:       16777216 kB   # Free swap
Dirty:            131072 kB   # Pages modified but not yet written to disk
Writeback:             0 kB   # Pages currently being written back
AnonPages:      28000000 kB   # Anonymous pages in userspace
Mapped:          4096000 kB   # Pages mapped into process address spaces
Shmem:           2048000 kB   # Shared memory (tmpfs, SHM)
KReclaimable:    3145728 kB   # Kernel memory that can be reclaimed under pressure
Slab:            4194304 kB   # Total slab allocations
SReclaimable:    3145728 kB   # Reclaimable slab (caches)
SUnreclaim:      1048576 kB   # Unreclaimable slab (kernel structures)
KernelStack:      131072 kB   # Kernel stacks
PageTables:       524288 kB   # Page table memory
CommitLimit:   131072000 kB   # Total committed memory allowed (with overcommit)
Committed_AS:   90000000 kB   # Currently committed virtual memory
HugePages_Total:    1024       # Huge pages configured
HugePages_Free:      512       # Huge pages available
HugePages_Rsvd:       64       # Reserved but not yet allocated
Hugepagesize:       2048 kB   # Size of each huge page
AnonHugePages:  10240000 kB   # Anonymous memory covered by THP

MemAvailable (not MemFree) is the right metric for “how much memory can a new process use?” It accounts for reclaimable page cache and slab caches, not just idle pages. MemFree of 512 MB on a 128 GB machine is normal and not a problem.

Dirty should be monitored. High dirty page counts cause write stalls when the kernel forces writeback. Related tunables:

1
2
3
4

# Current dirty limits
sysctl vm.dirty_ratio          # % of total RAM before hard throttle (default: 20)
sysctl vm.dirty_background_ratio  # % where background writeback starts (default: 10)
sysctl vm.dirty_expire_centisecs  # How long dirty pages wait before writeback (default: 3000 = 30s)

For write-heavy workloads (databases, logging), lowering dirty_background_ratio to 5% and dirty_ratio to 10% reduces I/O burst size and smooths write latency.

Memory Overcommit

By default, Linux allows processes to mmap() or malloc() more memory than physically exists. This works because:

1. Virtual pages are only backed by physical frames on first write (copy-on-write)
2. fork() creates a copy of the parent’s address space without copying pages

The vm.overcommit_memory sysctl controls the policy:

For databases and other memory-sensitive applications, overcommit mode 2 is safer — it prevents the OOM killer from terminating processes by refusing allocations upfront:

1
2
3

# Disable overcommit (conservative)
sysctl -w vm.overcommit_memory=2
sysctl -w vm.overcommit_ratio=80  # Use up to 80% of RAM + swap for commit limit

Huge Pages

The standard Linux page is 4 KB. The TLB covers a limited virtual address range — on a typical x86 CPU, the L1 TLB covers 32–64 entries × 4 KB = 128–256 KB. For a process with a 10 GB working set, TLB coverage is tiny and misses are frequent.

Huge pages solve this by using larger page sizes: 2 MB (hugepages) or 1 GB (gigantic pages) on x86-64. A 2 MB page covers 512× more address space per TLB entry.

Explicit Huge Pages

The traditional approach: pre-allocate a pool of 2 MB pages at boot and have applications request them explicitly via mmap(MAP_HUGETLB) or shmget(SHM_HUGETLB).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# Allocate 2048 × 2MB = 4 GB of huge pages
echo 2048 > /proc/sys/vm/nr_hugepages

# Or via sysctl
sysctl -w vm.nr_hugepages=2048

# Verify
grep HugePages /proc/meminfo
# HugePages_Total: 2048
# HugePages_Free:  2048
# Hugepagesize:    2048 kB

# Mount the hugetlbfs
mount -t hugetlbfs nodev /mnt/huge

In application code (C):

1
2
3
4
5
6

#include <sys/mman.h>

void *ptr = mmap(NULL, 2 * 1024 * 1024,
                 PROT_READ | PROT_WRITE,
                 MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB,
                 -1, 0);

Java, PostgreSQL, Oracle, and many other applications have explicit huge page support. For PostgreSQL:

1
2

# postgresql.conf
huge_pages = try   # or 'on' to require them

1
2
3

# Pre-allocate enough for shared_buffers
# If shared_buffers = 8GB, need at least 4096 × 2MB pages
sysctl -w vm.nr_hugepages=4096

Transparent Huge Pages (THP)

THP is the kernel’s automatic huge page promotion. It runs in the background, scans for naturally-aligned 2 MB regions of anonymous memory, and promotes them to huge pages without application changes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Check THP status
cat /sys/kernel/mm/transparent_hugepage/enabled
# [always] madvise never

# Options:
# always  - promote aggressively in background
# madvise - only for regions marked with madvise(MADV_HUGEPAGE)
# never   - disable THP

# Defrag behavior (how aggressively to defrag memory for THP)
cat /sys/kernel/mm/transparent_hugepage/defrag
# always defer defer+madvise [madvise] never

THP sounds like a pure win, but has real downsides:

The case against THP (always mode):

• Latency spikes: khugepaged (the THP daemon) does memory compaction that causes periodic stalls. Applications with strict p99 latency requirements (Redis, databases) often disable THP.
• Memory waste: a huge page is promoted for the whole 2 MB region even if only part of it is touched. Copy-on-write copies a whole 2 MB on fork instead of 4 KB.
• False sharing: two threads using different parts of a huge page share a TLB entry, potentially causing more TLB shootdowns.

Recommended configuration for latency-sensitive services:

1
2
3
4
5
6

# Use madvise mode: let applications opt in explicitly
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
echo defer+madvise > /sys/kernel/mm/transparent_hugepage/defrag

# For Redis specifically, disable entirely
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Recommended for throughput-oriented HPC / batch workloads:

1
2
3

# THP always mode with async defrag
echo always > /sys/kernel/mm/transparent_hugepage/enabled
echo defer > /sys/kernel/mm/transparent_hugepage/defrag

In your application, opt into THP selectively with madvise:

1
2
3
4
5
6
7

#include <sys/mman.h>

void *buf = mmap(NULL, size, PROT_READ|PROT_WRITE,
                  MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
madvise(buf, size, MADV_HUGEPAGE);   // request THP
// or
madvise(buf, size, MADV_NOHUGEPAGE); // opt out

1 GB Gigantic Pages

For very large working sets (ML inference engines, in-memory databases), 1 GB pages further reduce TLB pressure. Must be allocated at boot:

1
2
3
4
5
6
7

# Kernel command line (GRUB)
GRUB_CMDLINE_LINUX="hugepagesz=1G hugepages=32 default_hugepagesz=1G"

# Verify
grep Huge /proc/meminfo
# HugePages_Total:      32
# Hugepagesize:    1048576 kB

NUMA: Non-Uniform Memory Access

Modern multi-socket servers have NUMA architecture: each CPU socket has local memory with low latency (typically 80–100 ns), and remote memory attached to another socket with higher latency (typically 150–300 ns).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Check NUMA topology
numactl --hardware
# available: 2 nodes (0-1)
# node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 24 25 26 27 28 29 30 31 32 33 34 35
# node 0 size: 64512 MB
# node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23 36 37 38 39 40 41 42 43 44 45 46 47
# node 1 size: 64512 MB
# node distances:
# node   0   1
#   0:  10  21
#   1:  21  10

The distance matrix shows relative access costs. Local access = 10, remote = 21 (2.1× slower).

NUMA Allocation Policies

The kernel’s vm.zone_reclaim_mode and per-process NUMA policy control where memory is allocated:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Default policy: prefer local node, fall back to remote
numactl --localalloc ./myapp

# Interleave across nodes (good for apps that don't fit in one node's RAM)
numactl --interleave=all ./myapp

# Bind to specific node
numactl --membind=0 --cpunodebind=0 ./myapp

# Check NUMA stats
numastat
# Per-node allocation stats: numa_hit, numa_miss, numa_foreign, etc.

# Per-process NUMA stats
numastat -p $(pgrep myapp)

zone_reclaim_mode controls how aggressively the kernel reclaims local memory before going to a remote NUMA node:

1
2
3
4

# 0 = default: don't zone-reclaim, just go to remote node
# 1 = zone-reclaim before remote allocation
# 2 = write dirty pages to disk before remote allocation (aggressive)
sysctl vm.zone_reclaim_mode

For most workloads, zone_reclaim_mode=0 is correct. For workloads with a working set that fits in one node’s memory, zone_reclaim_mode=1 reduces remote accesses.

NUMA and the Linux Scheduler

The scheduler tracks per-task NUMA placement using NUMA balancing. It periodically unmaps pages and re-faults them on the node where the accessing CPU lives, gradually migrating memory to where it’s used.

1
2
3
4
5
6
7
8

# Enable/disable NUMA balancing
sysctl kernel.numa_balancing   # 0 or 1

# NUMA balancing stats
cat /proc/vmstat | grep numa
# numa_hit: pages allocated on the intended node
# numa_miss: pages allocated on the wrong node
# numa_page_migrate: pages migrated by NUMA balancing

For latency-sensitive applications, it can be better to pin processes to specific NUMA nodes with numactl and disable automatic NUMA balancing:

1

sysctl -w kernel.numa_balancing=0

The OOM Killer

When the system runs out of memory and can’t reclaim any, the Out-Of-Memory killer selects and terminates a process to free memory. Understanding how it selects victims — and how to tune it — is essential for production systems.

OOM Score

Every process has an oom_score (0–1000) that represents how “killable” it is. Higher score = more likely to be killed. The score is roughly proportional to the process’s RSS relative to total memory.

1
2
3

# Check a process's OOM score
cat /proc/$(pgrep postgres)/oom_score        # current computed score
cat /proc/$(pgrep postgres)/oom_score_adj    # adjustment (-1000 to +1000)

Tuning OOM Behavior

oom_score_adj: Adjust how likely a process is to be killed. Range: -1000 (never kill) to +1000 (kill first).

1
2
3
4
5

# Protect a critical process (like a monitoring agent)
echo -500 > /proc/$(pgrep prometheus)/oom_score_adj

# Make a disposable worker the first target
echo 500 > /proc/$(pgrep worker)/oom_score_adj

In systemd units:

1
2
3
4
5
6

# /etc/systemd/system/myapp.service
[Service]
OOMScoreAdjust=-500   # protect from OOM

# Or prefer to kill this service first
OOMScoreAdjust=500

vm.oom_kill_allocating_task: By default, the OOM killer selects the “worst” process. Set to 1 to kill the process that triggered the OOM instead (simpler, avoids killing important processes):

1

sysctl -w vm.oom_kill_allocating_task=1

vm.panic_on_oom: For high-availability systems where an OOM kill is unacceptable (database, control plane):

1

sysctl -w vm.panic_on_oom=1   # kernel panic on OOM (triggers watchdog reboot)

Cgroup Memory Limits

A better approach than relying on the OOM killer is to constrain processes with cgroup memory limits. When a cgroup hits its limit, only processes in that cgroup are OOM-killed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Create a cgroup (cgroup v2)
mkdir /sys/fs/cgroup/myapp

# Set memory limit (4 GB)
echo $((4 * 1024 * 1024 * 1024)) > /sys/fs/cgroup/myapp/memory.max

# Set memory+swap limit
echo $((4 * 1024 * 1024 * 1024)) > /sys/fs/cgroup/myapp/memory.swap.max

# Add a process
echo $$ > /sys/fs/cgroup/myapp/cgroup.procs

In systemd:

1
2
3

[Service]
MemoryMax=4G
MemorySwapMax=0   # disable swap for this service

In Kubernetes:

1
2
3
4
5

resources:
  limits:
    memory: "4Gi"  # sets cgroup memory.max
  requests:
    memory: "2Gi"

Reading OOM Kill Events

1
2
3
4
5
6
7
8
9

# See recent OOM kills in the kernel log
dmesg | grep -i "out of memory"
dmesg | grep -i "oom_kill"

# Structured OOM log entry shows:
# - Which process was killed
# - Its oom_score at kill time
# - Memory stats for all processes
# - Total memory state

Set up monitoring for OOM kills. The metric node_vmstat_oom_kill in the Prometheus node_exporter tracks this.

Memory Pressure and Swappiness

vm.swappiness

vm.swappiness (0–200, default 60) controls the kernel’s tendency to swap anonymous pages versus evicting file-backed page cache pages.

• swappiness = 0: avoid swapping anonymous pages until absolutely necessary (prefer to drop cache)
• swappiness = 60: balanced default
• swappiness = 100: treat anonymous pages and file cache equally
• swappiness = 200: aggressively swap anonymous pages even when cache could be evicted

For databases and caches that store their working set in anonymous memory, low swappiness is usually correct:

1

sysctl -w vm.swappiness=10   # for Redis, PostgreSQL, etc.

Memory Compaction

Over time, physical memory fragments — it’s hard to find contiguous ranges for huge page allocations. The kernel runs kcompactd to defragment:

1
2
3
4
5
6

# Trigger compaction manually
echo 1 > /proc/sys/vm/compact_memory

# View fragmentation state
cat /proc/buddyinfo
cat /sys/kernel/debug/extfrag/unusable_index  # fragmentation index per order

/proc/vmstat: Detailed Memory Statistics

1
2
3
4
5
6
7
8

# Key counters to monitor
cat /proc/vmstat | grep -E "pgfault|pgmajfault|pgscan|pgsteal|oom"

# pgfault: minor faults (page not in TLB but in RAM) — normal
# pgmajfault: major faults (page not in RAM, must read from disk) — expensive
# pgscan_kswapd: pages scanned by kswapd (background reclaim)
# pgsteal_kswapd: pages reclaimed by kswapd
# oom_kill: OOM kill events

High pgmajfault rates indicate your working set doesn’t fit in RAM. High pgscan with low pgsteal indicates memory pressure where the kernel is working hard to find pages to reclaim.

perf mem: Memory Access Profiling

perf mem uses hardware performance counters (specifically PEBS — Precise Event-Based Sampling) to sample memory accesses and identify cache misses, NUMA misses, and TLB misses.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Record memory access samples
sudo perf mem record -a -- sleep 10

# Or attach to a running process
sudo perf mem record -p $(pgrep myapp) -- sleep 10

# Analyze
sudo perf mem report

# Show only cache misses
sudo perf mem report --sort=mem,sym | head -40

Example output:

# Overhead  Samples  Local    Memory access
    24.50%       245  L3 miss  [k] __memcpy_avx_unaligned
    18.32%       183  L1 miss  [.] process_batch
    12.10%       121  LFB hit  [.] hash_lookup
     8.90%        89  L2 miss  [.] tree_walk
     5.60%        56  Remote   [.] global_counter_update

The “Remote” entries are NUMA remote accesses — these are expensive and often fixable with numactl.

perf stat for Memory Counters

1
2
3
4
5
6
7

# Hardware memory counters
sudo perf stat -e \
  cache-misses,cache-references,\
  LLC-load-misses,LLC-loads,\
  dTLB-load-misses,dTLB-loads,\
  mem-loads,mem-stores \
  -p $(pgrep myapp) -- sleep 10

Target ratios:

• LLC miss rate < 1%: good
• LLC miss rate 1–10%: investigate
• LLC miss rate > 10%: working set exceeds L3 cache, probably memory-bound

vmtouch: Working Set Analysis

vmtouch shows which files are currently in the page cache and allows you to warm the cache:

1
2
3
4
5
6
7
8

# Check what fraction of a file is in page cache
vmtouch /var/lib/postgresql/14/main/base/

# Warm the page cache for a database
vmtouch -t /var/lib/postgresql/14/main/base/

# Lock pages in cache (prevent eviction) — use carefully
vmtouch -l /hot/data/file.dat

Practical Tuning Scenarios

High-Throughput Web Server

1
2
3
4
5
6
7

# Maximize page cache hits, minimize swap
vm.swappiness=10
vm.dirty_ratio=10
vm.dirty_background_ratio=5

# Disable THP (latency spikes from compaction)
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

PostgreSQL Database

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Pre-allocate huge pages for shared_buffers
# shared_buffers = 32GB → need 16384 × 2MB pages
vm.nr_hugepages=16384

# Disable THP (PostgreSQL has its own buffer management)
echo never > /sys/kernel/mm/transparent_hugepage/enabled

# Reduce swappiness
vm.swappiness=1

# Protect from OOM
echo -900 > /proc/$(pgrep -f "postgres: checkpointer")/oom_score_adj

Redis

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Redis is famous for THP latency issues
echo never > /sys/kernel/mm/transparent_hugepage/enabled

# Redis will warn about this at startup if not set:
# "WARNING you have Transparent Huge Pages (THP) support enabled in your kernel"

# Low swappiness: Redis should fit in RAM
vm.swappiness=1

# Disable overcommit (Redis uses fork for persistence)
vm.overcommit_memory=1

ML Training (GPU + Large Batch)

1
2
3
4
5
6
7
8
9

# Enable THP for large tensor allocations
echo always > /sys/kernel/mm/transparent_hugepage/enabled
echo defer > /sys/kernel/mm/transparent_hugepage/defrag

# Pin process to NUMA node closest to GPU
numactl --cpunodebind=0 --membind=0 python train.py

# Pre-allocate huge pages for model weights
vm.nr_hugepages=65536  # 128 GB worth of 2MB pages

Quick Reference: Key Files and Commands

Memory management tuning is empirical — measure first, then tune. The tools above give you the observability to understand what the system is actually doing, which is the prerequisite for making it better.