There are such things as L1 switches, they are not really switches in the Ethernet sense but rather more like crossbar switches. They can selectively connect all traffic from two arbitrary ports together, and then change that on the fly, without a waiting change. Applications that are obsessed with getting the lowest possible latency might opt for those.