A GPU does not win at matrix multiplication because its cores are faster than a CPU's. They are slower, dumber, and clocked lower. It wins because it has thousands of them, schedules them in warps, and hides memory latency by oversubscribing the machine so aggressively that there is always work to run. Here is how SIMT, occupancy, coalescing, and tensor cores actually work.
Simt
-
Why GPUs Beat CPUs at Matrix Math