Networking: How Thousands of Accelerators Are Made to Look Like One

A frontier model does not fit on one accelerator. Its parameters are split across thousands, and after every training step each one has to share what it learned with the others — for months. That makes the network a first-order component rather than plumbing. A training step finishes when its slowest participant finishes, so the worst link governs the whole cluster, and an accelerator waiting on a gradient is exactly as idle as an accelerator nobody bought.

Two networks sit inside every such cluster, built on opposite principles. Scale-up joins a few dozen accelerators into something software treats as one machine, over a proprietary memory-semantic fabric — NVLink reaches 1.8 TB/s bidirectional per GPU across up to 576 of them. Scale-out joins thousands of those machines over a network assembled from parts many vendors sell. The boundary between the two is not physics; it is a judgement about how far the expensive fabric should reach, and it moves.

The specialised side of the argument is InfiniBand, and its record is a clean twenty-year doubling: 2 Gbit/s per lane at SDR in 2001, 200 at XDR in 2024, which on a four-lane port is 800 Gbit/s. But the number that explains its persistence is latency, not bandwidth — adapter latency fell from about 5 microseconds to under 0.6 — because a workload that synchronises thousands of processes at every step cares about the tail far more than the average.

Ethernet borrowed the trick. RDMA lets one machine write into another's memory without troubling the receiving processor, and RoCE brings it to Ethernet — maintained, revealingly, by the InfiniBand Trade Association. Version 2 runs over UDP and is routable, with latencies as low as 1.3 microseconds. The cost is the part the marketing leaves out: RoCE needs a network that does not drop packets, which means Priority Flow Control and a Data Center Bridging setup harder to configure than InfiniBand — and Priority Flow Control can deadlock the network outright.

So an industry body formed to fix Ethernet properly. The Ultra Ethernet Consortium names the deficiencies precisely: scale, bandwidth density, multi-pathing, fast reaction to congestion, and the inter-dependency between flows where tail latency is the figure of merit. Its steering members are AMD, Arista, Broadcom, Cisco, HPE, Intel, Meta, Microsoft and Oracle — with Google, NVIDIA, Qualcomm, Dell and Huawei among more than forty general members. The largest buyers of AI infrastructure are funding the commoditisation of one of its most profitable layers, which is what large customers facing one supplier have always done.

There is a precedent, and the underlying census turns out to complicate it. Tertiary accounts record InfiniBand leading the TOP500 interconnect ranking between 2014 and June 2016 before 10 Gigabit Ethernet displaced it. But the June 2026 list, read directly, shows InfiniBand on 292 of the 500 systems against Gigabit Ethernet's 165 — the most common interconnect on the list, not a displaced one. Measured by aggregate performance the ordering inverts: Ethernet accounts for about 8.54 exaflops against InfiniBand's 7.10. More machines run InfiniBand; more compute runs on Ethernet. Both halves are usually quoted separately, by people who prefer one of them.

The merchant vendors' answer is concrete rather than rhetorical. Arista, a consortium steering member, ships features that map one-for-one onto the deficiencies the consortium named: Multipath Reliable Connection and packet spraying for the missing multi-pathing, Congestion Signaling and PFC-aware load balancing with ECN for slow congestion reaction. It states platforms from 102.4 Tbps up to 460 Tbps, and a distributed switch supporting more than 30,000 400GbE accelerators as a single system — a claim that commodity Ethernet can be made to behave as one fabric at frontier-training scale, which is exactly what a proprietary fabric exists to deny.

Meanwhile the physical limit arrives. Broadcom's merchant Tomahawk 5 states 51.2 Tb/s of switching capacity; NVIDIA states 102.4 Tb/s per chip with 512 ports of 800 gigabits in five rack units. At that density, getting the signals off the chip at all is the constraint — which is where co-packaged optics stops being a research topic and becomes the only remaining option.

What is missing is named rather than implied. The Ultra Ethernet specification itself sits behind a download form and was not read, so this page describes the standard's intent and not its content. IEEE 802.3df, which defines the 800 Gb/s and 1.6 Tb/s rates quoted here from vendor pages, is paywalled. NVIDIA's own InfiniBand product pages were not retrieved, so the InfiniBand material is encyclopedic rather than first-hand. Arista's AI networking position was not read, which means this collection carries the integrated vendor's diagnosis of Ethernet's shortcomings without the merchant vendors' reply. And the TOP500 statistics behind the displacement story were not retrieved, so that history is summarised rather than shown.