LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Thunderbolt and USB4: How Tunneling Actually Works

thunderboltusb4pciehardwareconnectivitysignaling

For thirty years a cable did exactly one thing. A VGA cable carried analog video and nothing else; a USB cable carried USB and nothing else; an Ethernet cable carried Ethernet. The protocol and the physical wire were welded together, and the connector on the end told you precisely what was inside. Thunderbolt and USB4 broke that assumption, and almost nobody noticed because the connector — USB-C — looks like just another port. What is actually happening inside a USB4 link is closer to a tiny packet-switched network than a cable: the wire carries an abstract, multiplexed fabric, and the real protocols you care about — PCIe, DisplayPort, USB3 — ride across it as tunnels, encapsulated into fabric packets and reassembled transparently at the far end. Understanding that one shift, from dedicated wire to tunneled fabric, explains everything confusing about modern connectivity: why one port can drive a monitor and a GPU and a flash drive at once, why two identical-looking USB-C ports have wildly different capabilities, and why plugging in a dock can, in principle, read your encryption keys out of RAM.


From Light Peak to a Donated Standard

Thunderbolt began as an Intel research project called Light Peak, conceived as an optical interconnect — the original demos ran over fiber. Economics intervened, as they tend to: copper was cheap and good enough at the target distances, so the shipping product in 2011 was electrical. Thunderbolt 1 and 2 used the Mini DisplayPort connector and combined two things that would define the whole lineage — PCIe for data and DisplayPort for video, multiplexed over one link. Thunderbolt 3 made the pivotal move to the USB-C connector, pushed the link to 40 Gbps, and tunneled PCIe 3.0 x4 plus DisplayPort plus, now, USB. That combination — a general-purpose external PCIe bus and a display output sharing a reversible connector — is what made docks, eGPUs, and single-cable workstations possible.

The crucial political event came next. Intel donated the Thunderbolt 3 protocol to the USB Implementers Forum, and the USB-IF used it as the foundation for USB4. This is why USB4 and Thunderbolt are not rivals so much as siblings sharing a skeleton: USB4 is the Thunderbolt 3 architecture, standardized and made royalty-free, with some features the spec lets a vendor omit. USB4 Version 1.0 matched Thunderbolt’s 40 Gbps. The latest step — USB4 Version 2.0 and its certified, premium sibling Thunderbolt 5 — doubles the wire to 80 Gbps and adds a dynamic asymmetric mode reaching 120 Gbps in one direction. The names diverge but the machinery is shared, and the machinery is the interesting part.


The Core Idea: A Cable That Carries a Fabric

A traditional cable is a dedicated pipe. A USB4 link is a shared medium with a scheduler. Instead of the wire being PCIe or being DisplayPort, the wire carries a stream of tunnel packets, and each native protocol is chopped up, wrapped in those packets, routed across the link, and unwrapped on the other side. To the PCIe device, it still looks like it is sitting on a local PCIe bus. To the monitor, it still looks like a direct DisplayPort connection. Neither endpoint knows it is being tunneled — that transparency is the entire point.

Three protocols get tunneled, and they have very different temperaments:

Tunnel What it carries Timing requirement Why it matters
DisplayPort Video main stream + AUX Isochronous — fixed, jitter-intolerant A dropped or late frame is visible; bandwidth must be reserved up front
PCIe PCIe TLPs (transaction layer packets) Low-latency, bursty Turns the cable into an external PCIe slot: eGPUs, NVMe, capture cards
USB3 USB 3.x packets Bursty, tolerant Backward compatibility with the entire USB device universe

DisplayPort is the diva. Video is isochronous: the pixels for the next frame must arrive on time, every time, because there is no buffer deep enough to hide a stall and no way to retransmit a late frame the viewer would not see as a glitch. So the fabric reserves DisplayPort bandwidth first, as a guaranteed allocation, before anything else gets scheduled. PCIe and USB3 are more relaxed — they tolerate jitter and retransmission — so they divide up whatever bandwidth remains. This is why a setup driving two high-refresh displays leaves less headroom for your external SSD: the displays got their reservation off the top.


Inside the Box: Routers, Adapters, and Paths

The abstraction that makes tunneling work is that a USB4 domain is built from routers. Your laptop has a host router; a dock or hub has a device router; the chain of them forms a tree. They are identified and addressed by a topology ID — a route string that names a path through the tree, exactly the way a network addresses a node. The PCIe fabric thinking from the systems-engineer’s view is the right mental model: this is a switched fabric, not a bus, and the routers are its switches.

Each router is built from adapters and an internal transport layer. Two kinds of adapter matter:

  • Lane adapters face outward — they are the physical USB4 ports, driving the actual wire.
  • Protocol adapters face inward — a DisplayPort-IN adapter, a PCIe adapter, a USB3 adapter. These are the encapsulation engines: a protocol adapter takes native traffic, wraps it into tunnel packets, and hands it to the transport layer.

A tunnel is set up as a path: a reserved route from a source protocol adapter, through one or more routers, to a destination protocol adapter, with buffer space allocated at each hop. The transport layer moves tunnel packets along paths using credit-based flow control — a receiver advertises how many buffer credits (how much free buffer) it has, and a sender may only transmit while it holds credits. This is the same backpressure mechanism that keeps any packet fabric from overflowing its buffers; a USB3 adapter, for instance, will queue a tunneled packet rather than drop it when credits run short, because silently discarding it would corrupt the tunneled stream. Setting all of this up — discovering the topology, deciding which tunnels to create, reserving DisplayPort bandwidth, allocating credits — is the job of a software component in the OS called the Connection Manager. The hardware provides the fabric; the Connection Manager is the control plane that programs it.

  Laptop (Host Router)                    Dock (Device Router)
  ┌───────────────────────┐               ┌────────────────────────┐
  │  ┌─────────────────┐  │               │  ┌──────────────────┐  │
  │  │ DP-IN adapter   │──┼──┐         ┌──┼─▶│ DP-OUT ─▶ Monitor│  │
  │  │ PCIe adapter    │──┼─ │ tunnels │ ─┼─▶│ PCIe  ─▶ NVMe    │  │
  │  │ USB3 adapter    │──┼──┘  share  └──┼─▶│ USB3  ─▶ keyboard│  │
  │  └────────┬────────┘  │  one  link    │  └──────────────────┘  │
  │   Transport layer     │               │   Transport layer      │
  │   (paths + credits)   │               │                        │
  │  ┌────────▼────────┐  │  USB4 link    │  ┌──────────────────┐  │
  │  │  Lane adapter   │◀─┼═══════════════┼─▶│  Lane adapter    │  │
  │  └─────────────────┘  │ 1-2 lanes ea. │  └──────────────────┘  │
  └───────────────────────┘  direction    └────────────────────────┘
        DisplayPort reserves bandwidth first; PCIe + USB3 split the rest

How 80 and 120 Gigabits Happen: PAM3 and Asymmetry

Doubling a serial link’s bandwidth the naive way means doubling the clock, which roughly doubles the signal frequency, which makes the wire harder to drive, the cable shorter, and the power higher. Thunderbolt 5 and USB4 v2 mostly sidestepped that by changing the encoding instead of the clock. Earlier generations used NRF/NRZ-style binary signaling: each symbol on the wire is one of two voltage levels, carrying one bit. TB5 switched to PAM3 — pulse-amplitude modulation with three levels (think −1, 0, +1). Three levels per symbol carries roughly 1.58 bits instead of 1, so you move about 1.58× more data per symbol without proportionally raising the frequency. That encoding trick, more than raw clock speed, is what gets the link from 40 to 80 Gbps. The same signal-integrity pressures that make Cat6 twist its pairs drive these choices: at these rates the analog reality of the copper, not the digital ideal, sets the ceiling.

The second trick is asymmetric mode, and it is genuinely clever. A USB4 link has lanes in each direction. Thunderbolt 5 can dynamically reconfigure them so that three lanes’ worth of bandwidth runs one way and one lane runs the other — 120 Gbps transmit, 40 Gbps receive — and switch back to symmetric 80/80 on demand. Why would you want a lopsided link? Because some workloads are lopsided. Driving multiple high-resolution, high-refresh displays is overwhelmingly a transmit problem: the host pushes enormous pixel streams out and gets almost nothing back. An eGPU rendering to external panels wants exactly this shape. The fabric notices the demand and re-allocates the wire’s capacity to match, which is something a fixed-function cable could never do because it had no concept of bandwidth as a reallocatable pool.

Generation Max bandwidth Signaling PCIe tunnel Connector
Thunderbolt 3 40 Gbps NRZ PCIe 3.0 x4 (~32 Gbps) USB-C
USB4 v1 / Thunderbolt 4 40 Gbps NRZ PCIe 3.0 x4 (TB4 mandatory) USB-C
USB4 v2 / Thunderbolt 5 80 Gbps sym, 120/40 asym PAM3 PCIe 4.0 x4 (~64 Gbps) USB-C

PCIe Tunneling Is the Superpower and the Liability

Of the three tunnels, PCIe is the one that turns a cable into something new. Tunneling DisplayPort gets you a monitor; tunneling USB3 gets you the existing USB world; but tunneling PCIe gets you an external slot on the system bus. That is what an eGPU is — a graphics card sitting in an enclosure, talking PCIe through the cable as though it were plugged into the motherboard. Thunderbolt 3 tunneled PCIe 3.0 x4, around 32 Gbps; Thunderbolt 5 tunnels PCIe 4.0 x4, around 64 Gbps, which is why TB5 is the first generation where an external GPU stops being badly bottlenecked. NVMe enclosures, capture cards, and audio interfaces all ride the same PCIe tunnel.

But PCIe is a bus designed on the assumption that everything on it is trusted, because historically everything on it was soldered inside the chassis. A PCIe device can perform DMA — direct memory access — reading and writing system RAM without going through the CPU. That is a performance necessity for a GPU and a catastrophe for a thing you plug in at a coffee shop. The Thunderspy and Thunderclap research demonstrated exactly this: a malicious Thunderbolt device tunneling PCIe could reach into host memory and exfiltrate secrets or implant code, bypassing the lock screen entirely. The defense is the IOMMU (Intel VT-d / AMD-Vi), which sits between tunneled PCIe devices and RAM and remaps their DMA so a device can only touch memory explicitly granted to it. On top of that, Thunderbolt defines security levels and OS-level device authorization, so a newly attached PCIe-tunneling device is not trusted until the user approves it:

Security level Behavior
SL0 No security — any device works (legacy, dangerous)
SL1 User must authorize each new device before it connects
SL2 Authorization with a saved cryptographic key per device
SL3 DisplayPort and USB only — PCIe tunneling disabled entirely

Modern operating systems pair these with Kernel DMA Protection, which keeps the IOMMU active for external ports from boot. The honest takeaway: PCIe tunneling is both the reason Thunderbolt is exciting and the reason it needs more security machinery than any other consumer port. If you do not need external PCIe — if you only ever attach displays and USB devices — SL3 is a defensible posture.


Why Two Identical Ports Behave Differently

Here is the practical confusion the tunneling model creates. The USB-C connector is just a shape. What flows through it depends on which optional features the silicon behind it implements, and USB4 makes several of the important ones optional. PCIe tunneling, in particular, is mandatory for Thunderbolt certification but optional for plain USB4. So you can have two laptops, each with a USB-C port labeled “USB4,” where one drives an eGPU and the other simply cannot — because the second one’s USB4 controller skipped PCIe tunneling. Nothing on the outside tells you. Display capabilities, the number of supported tunnels, the bandwidth tier (40 vs 80 Gbps), and power delivery are all independently variable behind the same physical jack.

This is where the certification programs diverge in value. Thunderbolt is Intel’s brand, and Thunderbolt 4 and 5 certification mandate a floor: PCIe tunneling present, a minimum number of displays, a minimum PCIe bandwidth, mandatory testing against a compatibility suite. USB4 is the open standard with a wider menu of optional features, which is great for cost-sensitive devices but terrible for the buyer trying to predict behavior from the label. The rule of thumb: the Thunderbolt logo is a guarantee of a capability set; the USB4 logo is a guarantee of a protocol skeleton whose features you must look up. Power, incidentally, is not tunneled — USB Power Delivery negotiates separately over the connector’s configuration channel, which is why charging and data capability are also independent of each other.

For the practically minded, Linux exposes the whole fabric, and inspecting it demystifies a lot:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# Show the Thunderbolt/USB4 domain, routers, and authorized devices
boltctl list

# Kernel view of the same tree (security level, authorization state)
ls /sys/bus/thunderbolt/devices/
cat /sys/bus/thunderbolt/devices/domain0/security   # secure / user / dponly / none

# Authorize a connected device (when security level requires it)
boltctl authorize <uuid>

# Confirm the IOMMU is actually on (the DMA defense)
dmesg | grep -iE "DMAR|IOMMU|VT-d"

# See tunneled PCIe devices appear on the bus when a dock/eGPU attaches
lspci -tv

# Check negotiated link: lanes, gen, symmetric vs asymmetric (newer tooling)
boltctl domains

Cables, Distance, and the Limits of Copper

Tunneling does not repeal physics. At 40 Gbps a passive USB-C cable is limited to about 0.8 m before signal integrity collapses; beyond that you need an active cable with retimer chips that regenerate the signal, or an optical cable that converts to light for the long run. Thunderbolt 5’s 80 and 120 Gbps modes tighten this further — the faster the symbols, the shorter the passive reach — which is why TB5 cables are short, thick, and certified, and why a random USB-C cable from a drawer may negotiate down to a slow, charging-only link without telling you why. The cable is now an active participant with its own capabilities, not a dumb wire, and a mismatch between cable, host, and device silently drops you to the lowest common denominator all three support. This is the same lesson the PCIe physical layer teaches and that the CXL fabric inherits: at multi-gigabit rates, the link is an analog negotiation pretending to be a digital certainty.


Verdict

Thunderbolt and USB4 are best understood not as faster cables but as a small switched fabric hiding inside a connector. The decisive idea is tunneling: native PCIe, DisplayPort, and USB3 traffic encapsulated into fabric packets, routed across a tree of routers by a credit-managed transport layer, and unwrapped transparently at the far end, with DisplayPort reserving its bandwidth first because video cannot wait and the rest of the link divided among the protocols that can. That architecture is what lets a single port simultaneously drive displays, an external GPU, and a flash drive, and it is what PAM3 signaling and Thunderbolt 5’s asymmetric 120 Gbps mode push to ever-higher ceilings. The same flexibility creates the two recurring frustrations: PCIe tunneling delivers eGPU-grade performance but opens a DMA attack surface that demands an IOMMU and per-device authorization, and the optional-feature menu behind the universal USB-C jack means the label on a port tells you almost nothing about what it can actually do. The pragmatic stance is to treat the Thunderbolt logo as a capability guarantee, plain USB4 as a spec sheet you must read, SL3 as a reasonable default if you never tunnel PCIe, and the cable as a component with real limits rather than an afterthought. The cable that did one thing is gone; what replaced it is a network you can hold in your hand.


Sources

Comments