Why the network, not the GPU, decides how big a model you can train: the NVLink and NVSwitch scale-up domain versus the InfiniBand and RoCE scale-out fabric, rail-optimized topology, in-network reduction, and the bandwidth-budget math that maps tensor, pipeline, data, and expert parallelism onto real cables.
Roce
-
Training Cluster Networking: NVLink, Rails, and the Bandwidth Budget That Bounds Model Scale -
RDMA and InfiniBand From the Ground Up: Why It's Fast and How to Diagnose It Why RDMA and InfiniBand deliver 400 Gbps at single-microsecond latency with the CPU idle — how kernel-bypass and the verbs model work, how RoCE differs, and how to diagnose a fabric that isn't hitting its datasheet.