Why the network, not the GPU, decides how big a model you can train: the NVLink and NVSwitch scale-up domain versus the InfiniBand and RoCE scale-out fabric, rail-optimized topology, in-network reduction, and the bandwidth-budget math that maps tensor, pipeline, data, and expert parallelism onto real cables.
Rail-Optimized
-
Training Cluster Networking: NVLink, Rails, and the Bandwidth Budget That Bounds Model Scale