{"schema_version":"2026-09-05.topic-graph-v1","canonical_url":"https://www.manufacturing.ai/topics/networking","topic":{"slug":"networking","name":"Networking: How Thousands of Accelerators Are Made to Look Like One","description":"A sourced reference collection on the AI interconnect — why a training step finishes only when its slowest link does, the twenty-year doubling of InfiniBand, what RDMA over Ethernet actually costs to run, and why the biggest buyers of AI infrastructure are funding a consortium to commoditise it.","coverage_notes":"Seventh packet, and the component that turns the other six into a machine. Sixteen sources were consulted: seven were retrieved and read, three were attempted and could not be read, and six were surfaced and set aside with a stated reason. Two gaps named in the first revision are now closed, and both changed what this page says. The TOP500 interconnect census was read directly — it sits behind a category form, so a browser was required — and it complicates the tidy displacement story drawn from tertiary sources: on the June 2026 list InfiniBand leads by system count, 292 of 500, while Ethernet leads on aggregate performance at roughly 8.54 exaflops to 7.10. Publishing both orderings together is the point, because each is routinely quoted alone by a party that prefers it, and the assertion that previously carried only the displacement half has been superseded rather than quietly edited. Arista's position was also retrieved, so this collection no longer carries an integrated vendor's diagnosis of Ethernet's shortcomings without a merchant vendor's reply — and that reply maps feature-for-feature onto the deficiencies the Ultra Ethernet Consortium named. The strongest source remains the consortium's own statement of what Ethernet lacks, because an industry body naming deficiencies in the technology it champions is more credible than a vendor naming them in a competitor's. Vendor pages are used only for facts about their own products; every comparative multiple — NVIDIA's 1.6x network performance and 1.9x NCCL claims in particular — is recorded as a claim and asserted nowhere. Remaining named gaps: the Ultra Ethernet specification, behind a download form, so this page describes the standard's intent rather than its content; IEEE 802.3df, paywalled, so the 800 Gb/s and 1.6 Tb/s rates rest on vendor pages; and NVIDIA's InfiniBand product page, which was retrieved but states no port counts, bandwidths or latencies. Not yet editor-reviewed; every assertion reads as reported.","primer":"A frontier model does not fit on one accelerator. Its parameters are split across thousands, and after every training step each one has to share what it learned with the others — for months. That makes the network a first-order component rather than plumbing. A training step finishes when its slowest participant finishes, so the worst link governs the whole cluster, and an accelerator waiting on a gradient is exactly as idle as an accelerator nobody bought.\n\nTwo networks sit inside every such cluster, built on opposite principles. Scale-up joins a few dozen accelerators into something software treats as one machine, over a proprietary memory-semantic fabric — NVLink reaches 1.8 TB/s bidirectional per GPU across up to 576 of them. Scale-out joins thousands of those machines over a network assembled from parts many vendors sell. The boundary between the two is not physics; it is a judgement about how far the expensive fabric should reach, and it moves.\n\nThe specialised side of the argument is InfiniBand, and its record is a clean twenty-year doubling: 2 Gbit/s per lane at SDR in 2001, 200 at XDR in 2024, which on a four-lane port is 800 Gbit/s. But the number that explains its persistence is latency, not bandwidth — adapter latency fell from about 5 microseconds to under 0.6 — because a workload that synchronises thousands of processes at every step cares about the tail far more than the average.\n\nEthernet borrowed the trick. RDMA lets one machine write into another's memory without troubling the receiving processor, and RoCE brings it to Ethernet — maintained, revealingly, by the InfiniBand Trade Association. Version 2 runs over UDP and is routable, with latencies as low as 1.3 microseconds. The cost is the part the marketing leaves out: RoCE needs a network that does not drop packets, which means Priority Flow Control and a Data Center Bridging setup harder to configure than InfiniBand — and Priority Flow Control can deadlock the network outright.\n\nSo an industry body formed to fix Ethernet properly. The Ultra Ethernet Consortium names the deficiencies precisely: scale, bandwidth density, multi-pathing, fast reaction to congestion, and the inter-dependency between flows where tail latency is the figure of merit. Its steering members are AMD, Arista, Broadcom, Cisco, HPE, Intel, Meta, Microsoft and Oracle — with Google, NVIDIA, Qualcomm, Dell and Huawei among more than forty general members. The largest buyers of AI infrastructure are funding the commoditisation of one of its most profitable layers, which is what large customers facing one supplier have always done.\n\nThere is a precedent, and the underlying census turns out to complicate it. Tertiary accounts record InfiniBand leading the TOP500 interconnect ranking between 2014 and June 2016 before 10 Gigabit Ethernet displaced it. But the June 2026 list, read directly, shows InfiniBand on 292 of the 500 systems against Gigabit Ethernet's 165 — the most common interconnect on the list, not a displaced one. Measured by aggregate performance the ordering inverts: Ethernet accounts for about 8.54 exaflops against InfiniBand's 7.10. More machines run InfiniBand; more compute runs on Ethernet. Both halves are usually quoted separately, by people who prefer one of them.\n\nThe merchant vendors' answer is concrete rather than rhetorical. Arista, a consortium steering member, ships features that map one-for-one onto the deficiencies the consortium named: Multipath Reliable Connection and packet spraying for the missing multi-pathing, Congestion Signaling and PFC-aware load balancing with ECN for slow congestion reaction. It states platforms from 102.4 Tbps up to 460 Tbps, and a distributed switch supporting more than 30,000 400GbE accelerators as a single system — a claim that commodity Ethernet can be made to behave as one fabric at frontier-training scale, which is exactly what a proprietary fabric exists to deny.\n\nMeanwhile the physical limit arrives. Broadcom's merchant Tomahawk 5 states 51.2 Tb/s of switching capacity; NVIDIA states 102.4 Tb/s per chip with 512 ports of 800 gigabits in five rack units. At that density, getting the signals off the chip at all is the constraint — which is where co-packaged optics stops being a research topic and becomes the only remaining option.\n\nWhat is missing is named rather than implied. The Ultra Ethernet specification itself sits behind a download form and was not read, so this page describes the standard's intent and not its content. IEEE 802.3df, which defines the 800 Gb/s and 1.6 Tb/s rates quoted here from vendor pages, is paywalled. NVIDIA's own InfiniBand product pages were not retrieved, so the InfiniBand material is encyclopedic rather than first-hand. Arista's AI networking position was not read, which means this collection carries the integrated vendor's diagnosis of Ethernet's shortcomings without the merchant vendors' reply. And the TOP500 statistics behind the displacement story were not retrieved, so that history is summarised rather than shown.","primer_note":"Manufacturing.ai's own editorial synthesis, not evidence. Every factual statement in it is separately asserted and sourced in this response's concepts and events.","reviewed_through_date":null,"source_count":8,"inspected_source_count":8,"consulted_reference_count":9,"blocked_reference_count":3,"set_aside_reference_count":6},"figures":[{"kind":"time-series","title":"Twenty years of doubling","caption":"InfiniBand per-lane throughput by generation. Ports are usually four lanes wide, so the top of this chart is an 800 Gbit/s link.","sourceNote":"Wikipedia's InfiniBand article, read directly — a tertiary account of a generation table no vendor publishes neutrally. Stated per-lane throughput, not measured results.","unit":"Throughput per lane, Gbit/s","points":[{"label":"SDR 2001","value":2,"display":"2"},{"label":"DDR 2005","value":4,"display":"4"},{"label":"QDR 2007","value":8,"display":"8"},{"label":"FDR 2011","value":13.64,"display":"13.6"},{"label":"EDR 2014","value":25,"display":"25"},{"label":"HDR 2018","value":50,"display":"50"},{"label":"NDR 2022","value":100,"display":"100"},{"label":"XDR 2024","value":200,"display":"200"}]},{"kind":"composition","title":"What the top 500 machines actually run","caption":"Interconnect family by system count, June 2026. InfiniBand leads here — but on aggregate performance the ordering reverses and Ethernet leads.","sourceNote":"TOP500 List Statistics, June 2026 list, read through the site's own category form. Counts sum to exactly 500. A twice-yearly self-reported census of machines whose operators choose to submit, not a market survey.","totalDisplay":"500 systems","parts":[{"label":"InfiniBand","value":292,"display":"292"},{"label":"Gigabit Ethernet","value":165,"display":"165"},{"label":"Omnipath","value":26,"display":"26"},{"label":"Proprietary network","value":7,"display":"7"},{"label":"Custom interconnect","value":6,"display":"6"},{"label":"Ethernet","value":4,"display":"4"}]},{"kind":"time-series","title":"The latency that mattered more than the bandwidth","caption":"InfiniBand adapter latency, SDR to HDR. For a workload that synchronises thousands of processes every step, this fell further than bandwidth rose in importance.","sourceNote":"Wikipedia's InfiniBand article, read directly. It gives adapter latency of 5 microseconds at SDR and under 0.6 at HDR; the HDR figure is an upper bound, plotted at 0.6.","unit":"Adapter latency, microseconds","points":[{"label":"SDR","value":5,"display":"5 µs"},{"label":"HDR","value":0.6,"display":"<0.6 µs"}]}],"blocked_references":[{"title":"All AI Data Center Interconnects Will Be Optical Within 5 Years","publisher":"Semiconductor Engineering","url":"https://semiengineering.com/all-ai-data-center-interconnects-will-be-optical-within-5-years/","source_type":"journalism","retrieval_status":"blocked_403","content_inspected":false,"published_at":null},{"title":"Broadcom Announces Tomahawk 6 – Davisson (investor relations original)","publisher":"Broadcom Inc.","url":"https://investors.broadcom.com/news-releases/news-release-details/broadcom-announces-tomahawkr-6-davisson-industrys-first-1024","source_type":"press_release","retrieval_status":"timeout","content_inspected":false,"published_at":null},{"title":"Where co-packaged optics (CPO) technology stands in 2026","publisher":"EDN","url":"https://www.edn.com/where-co-packaged-optics-cpo-technology-stands-in-2026/","source_type":"journalism","retrieval_status":"timeout","content_inspected":false,"published_at":null}],"set_aside_references":[{"title":"IEEE 802.3df-2024: 200 Gb/s, 400 Gb/s, 800 Gb/s and 1.6 Tb/s Ethernet","publisher":"IEEE Standards Association","url":"https://standards.ieee.org/ieee/802.3df/10424/","source_type":"other","retrieval_status":"search_result_only","content_inspected":false},{"title":"NVIDIA Blackwell Architecture Explained: B200, GB200 & PCB Design Impact","publisher":"NextPCB","url":"https://www.nextpcb.com/blog/nvidia-blackwell-architecture-b200-gb200-pcb-design","source_type":"other","retrieval_status":"search_result_only","content_inspected":false},{"title":"NVIDIA GPU History: GeForce 256 to Vera Rubin","publisher":"IOH Tech","url":"https://iohtechco.com/resources/nvidia-gpu-history-evolution","source_type":"other","retrieval_status":"search_result_only","content_inspected":false},{"title":"NVIDIA Quantum InfiniBand","publisher":"NVIDIA","url":"https://www.nvidia.com/en-us/networking/products/infiniband/","source_type":"company_website","retrieval_status":"search_result_only","content_inspected":false},{"title":"Ultra Ethernet Specification 1.0.3","publisher":"Ultra Ethernet Consortium","url":"https://ultraethernet.org/specifications/","source_type":"other","retrieval_status":"search_result_only","content_inspected":false},{"title":"What is CUDA? Parallel programming for GPUs","publisher":"InfoWorld","url":"https://www.infoworld.com/article/2256401/what-is-cuda-parallel-programming-for-gpus.html","source_type":"journalism","retrieval_status":"search_result_only","content_inspected":false}],"concepts":[{"slug":"co-packaged-optics","name":"Co-Packaged Optics (CPO)","category":"packaging_technology","canonical_url":"https://www.manufacturing.ai/topics/networking#co-packaged-optics","assertions":[]},{"slug":"gpu","name":"Graphics Processing Unit (GPU)","category":"component","canonical_url":"https://www.manufacturing.ai/topics/networking#gpu","assertions":[]},{"slug":"infiniband","name":"InfiniBand","category":"interconnect","canonical_url":"https://www.manufacturing.ai/topics/networking#infiniband","assertions":[{"id":"066eb6b0-e654-4ff7-961f-6f488aa4d8ff","predicate":"concept.description","statement":"On the TOP500 list of June 2026, InfiniBand leads by system count with 292 of 500 against Gigabit Ethernet's 165 — but Ethernet leads on aggregate performance, 8,538,847,560 GFlops of Rmax against InfiniBand's 7,097,006,840. More machines run InfiniBand; more compute runs on Ethernet.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"71b8c88a-3fd3-4d0f-b779-b7fbb6ec92d0","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"TOP500 List Statistics — Interconnect Family, June 2026 list","publisher":"TOP500","url":"https://www.top500.org/statistics/list/","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":"2026-06-01","documents":[]}}]},{"id":"826a3521-ced8-45b6-bf46-557835fd8826","predicate":"concept.description","statement":"Tertiary accounts record InfiniBand leading the TOP500 interconnect share from 2014 until June 2016 before 10 Gigabit Ethernet displaced it — but the June 2026 census has InfiniBand leading by system count again, with Ethernet ahead only on aggregate performance.","assessment":"disputed","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"a12e1f6d-f7c0-4f86-841c-c057d61199e4","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"InfiniBand","publisher":"Wikipedia","url":"https://en.wikipedia.org/wiki/InfiniBand","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}},{"id":"057eacd3-f0b2-4b63-a309-da8ffefefdfc","stance":"challenges","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"TOP500 List Statistics — Interconnect Family, June 2026 list","publisher":"TOP500","url":"https://www.top500.org/statistics/list/","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":"2026-06-01","documents":[]}}]},{"id":"95b9435a-1142-4cad-a94c-7d48924b20f5","predicate":"concept.description","statement":"InfiniBand doubled roughly every generation from 2 Gbit/s per lane at SDR in 2001 to 200 at XDR in 2024 — 800 Gbit/s over a four-lane port — while adapter latency fell from about 5 microseconds to under 0.6, which is the figure that explains its persistence.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"dd23e284-4445-4c42-9deb-e7d0829027ae","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"InfiniBand","publisher":"Wikipedia","url":"https://en.wikipedia.org/wiki/InfiniBand","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"ethernet-for-ai","name":"Making Ethernet Fit for AI","category":"interconnect","canonical_url":"https://www.manufacturing.ai/topics/networking#ethernet-for-ai","assertions":[{"id":"7371065b-d476-4990-897e-b4f1f1267b7c","predicate":"concept.description","statement":"The Ultra Ethernet Consortium's steering members are AMD, Arista, Broadcom, Cisco, HPE, Intel, Meta, Microsoft and Oracle, with Google, NVIDIA, Qualcomm, Dell and Huawei among 40-plus general members — the largest buyers funding commoditisation of a profitable layer, with the incumbent inside the body.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"a6b54cb1-531b-4592-b830-4ecfc77dd6b4","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"Ultra Ethernet Consortium","publisher":"Ultra Ethernet Consortium","url":"https://ultraethernet.org/","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]},{"id":"acaef0cd-4f09-4452-93d9-4a630d80837a","predicate":"concept.description","statement":"The Ultra Ethernet Consortium names what Ethernet lacks for AI: scale, bandwidth density, multi-pathing, fast congestion response, and flow inter-dependency where tail latency is the figure of merit — a training step finishing only when its slowest participant does.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"97d2835c-dd63-45e8-8fd9-9e98d6487cfd","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"Ultra Ethernet Consortium","publisher":"Ultra Ethernet Consortium","url":"https://ultraethernet.org/","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]},{"id":"bf04ee47-076a-4374-992b-85a915362a25","predicate":"concept.description","statement":"Arista, a consortium steering member, answers the named Ethernet deficiencies with Multipath Reliable Connection, packet spraying, Congestion Signaling and PFC-aware load balancing, and states platforms from 102.4 Tbps up to 460 Tbps and a distributed switch supporting over 30,000 400GbE accelerators as one system.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"70e3c0a3-64be-4b01-81f1-8ce3f94ec2c5","stance":"supports","is_primary_for_assertion":true,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"Arista AI Networking","publisher":"Arista Networks","url":"https://www.arista.com/en/solutions/ai-networking","source_type":"company_website","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"nvlink-scale-up-fabric","name":"NVLink and the Scale-Up Fabric","category":"interconnect","canonical_url":"https://www.manufacturing.ai/topics/networking#nvlink-scale-up-fabric","assertions":[]},{"slug":"pluggable-optical-transceiver","name":"Pluggable Optical Transceiver","category":"component","canonical_url":"https://www.manufacturing.ai/topics/networking#pluggable-optical-transceiver","assertions":[]},{"slug":"rdma-and-lossless-ethernet","name":"RDMA and Lossless Ethernet","category":"interconnect","canonical_url":"https://www.manufacturing.ai/topics/networking#rdma-and-lossless-ethernet","assertions":[{"id":"4534a0f3-b1da-4815-8cd7-a51d14805b17","predicate":"concept.description","statement":"RoCE gives Ethernet remote direct memory access — v1 confined to one broadcast domain, v2 routable over UDP port 4791, latencies as low as 1.3 microseconds — but requires Priority Flow Control and Data Center Bridging, which is harder to configure than InfiniBand and can deadlock the network.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"1c93eb0e-c39d-4e6d-880a-480c726206a2","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"RDMA over Converged Ethernet","publisher":"Wikipedia","url":"https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"scale-up-versus-scale-out","name":"Scale-Up Versus Scale-Out","category":"interconnect","canonical_url":"https://www.manufacturing.ai/topics/networking#scale-up-versus-scale-out","assertions":[{"id":"8d2cc826-aa10-4cc5-921e-46920d755fa4","predicate":"concept.description","statement":"Scale-up joins dozens of accelerators as one machine over a proprietary fabric at terabytes per second per device; scale-out joins thousands of machines over multi-vendor networking at gigabits per port — and the boundary between them is a commercial judgement, not a physical law.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"d55c4a4a-a75b-40fe-b08e-6cf4a011a6d6","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"NVIDIA Blackwell Platform Arrives to Power a New Era of Computing","publisher":"NVIDIA","url":"https://nvidianews.nvidia.com/news/nvidia-blackwell-platform-arrives-to-power-a-new-era-of-computing","source_type":"press_release","retrieval_status":"fetched","content_inspected":true,"published_at":"2024-03-18","documents":[]}},{"id":"fd199078-6b16-46ce-8b3a-83a0b3475e00","stance":"supports","is_primary_for_assertion":true,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"NVIDIA Spectrum-X Ethernet Networking Platform","publisher":"NVIDIA","url":"https://www.nvidia.com/en-us/networking/spectrumx/","source_type":"company_website","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"silicon-photonics","name":"Silicon Photonics","category":"component","canonical_url":"https://www.manufacturing.ai/topics/networking#silicon-photonics","assertions":[]},{"slug":"switch-silicon-capacity","name":"Switch Silicon Capacity","category":"component","canonical_url":"https://www.manufacturing.ai/topics/networking#switch-silicon-capacity","assertions":[{"id":"c97b5d60-4ab6-4e21-b023-c383e1a15087","predicate":"concept.description","statement":"Broadcom's merchant Tomahawk 5 states 51.2 Tb/s of switching capacity; NVIDIA states 102.4 Tb/s per switch chip on 200G SerDes with 512 ports of 800G in a 5U chassis — a density at which getting signals off the chip becomes the constraint.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"1499e9f0-0ba5-4904-88d4-9d40bc9b3d16","stance":"supports","is_primary_for_assertion":true,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"NVIDIA Spectrum-X Ethernet Networking Platform","publisher":"NVIDIA","url":"https://www.nvidia.com/en-us/networking/spectrumx/","source_type":"company_website","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}},{"id":"fd59c3a5-585c-49ae-baa2-c58ab8830720","stance":"supports","is_primary_for_assertion":true,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"BCM78900 Series (Tomahawk 5) Ethernet Switch","publisher":"Broadcom","url":"https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm78900-series","source_type":"company_website","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]}],"events":[{"slug":"top500-interconnect-census-june-2026","title":"The Census Says Both Sides Are Winning","category":"other","event_date":"2026-06-01","date_precision":"month","country":null,"region":null,"canonical_url":"https://www.manufacturing.ai/topics/networking#top500-interconnect-census-june-2026","assertions":[{"id":"b5badfc7-6aed-4fbb-922c-a7b6c536272e","predicate":"event.description","statement":"The June 2026 TOP500 list records InfiniBand on 292 of 500 systems against Gigabit Ethernet's 165, while Ethernet leads aggregate performance at 8,538,847,560 GFlops of Rmax to InfiniBand's 7,097,006,840 — the specialised interconnect on more machines, the commodity one under more compute.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"3155c09d-e702-444e-aaaa-f27e776edd93","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"TOP500 List Statistics — Interconnect Family, June 2026 list","publisher":"TOP500","url":"https://www.top500.org/statistics/list/","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":"2026-06-01","documents":[]}}]}]},{"slug":"infiniband-xdr-2024","title":"InfiniBand Reaches 200 Gigabits per Lane","category":"technology_generation_milestone","event_date":"2024-01-01","date_precision":"year","country":null,"region":null,"canonical_url":"https://www.manufacturing.ai/topics/networking#infiniband-xdr-2024","assertions":[{"id":"e24329ec-66cb-4a72-8440-123224178df8","predicate":"event.description","statement":"InfiniBand's XDR generation reached 200 Gbit/s per lane in 2024 — 800 Gbit/s over a four-lane port, and a hundredfold increase in per-lane throughput since 2001.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"5a48de66-b5de-41ff-ae97-8d1cf988c058","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"InfiniBand","publisher":"Wikipedia","url":"https://en.wikipedia.org/wiki/InfiniBand","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"infiniband-displaced-in-top500-2016","title":"Commodity Ethernet Displaces InfiniBand at the Top of the TOP500","category":"other","event_date":"2016-06-01","date_precision":"month","country":null,"region":null,"canonical_url":"https://www.manufacturing.ai/topics/networking#infiniband-displaced-in-top500-2016","assertions":[{"id":"573d3973-4903-44d3-8ec9-c64c1683f63a","predicate":"event.description","statement":"InfiniBand was the most common TOP500 supercomputer interconnect from 2014 until June 2016, when 10 Gigabit Ethernet displaced it — the clearest precedent for a specialised interconnect losing share to a commodity one.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"be7b25fe-8550-4b5e-b74a-49990858b377","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"InfiniBand","publisher":"Wikipedia","url":"https://en.wikipedia.org/wiki/InfiniBand","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"infiniband-sdr-2001","title":"InfiniBand Ships at 2 Gigabits per Lane","category":"standards_milestone","event_date":"2001-01-01","date_precision":"year","country":null,"region":null,"canonical_url":"https://www.manufacturing.ai/topics/networking#infiniband-sdr-2001","assertions":[{"id":"49428b84-2f36-4e1a-a5e5-1446e900b2ec","predicate":"event.description","statement":"InfiniBand's first generation shipped from 2001 at 2.5 Gbit/s signalling and 2 Gbit/s throughput per lane, with adapter latency around 5 microseconds.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"a48e2878-69f5-4189-9689-f80139f85c46","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"InfiniBand","publisher":"Wikipedia","url":"https://en.wikipedia.org/wiki/InfiniBand","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"ultra-ethernet-specification-published","title":"An Industry Consortium Publishes a Specification for AI Ethernet","category":"standards_milestone","event_date":null,"date_precision":"unknown","country":null,"region":null,"canonical_url":"https://www.manufacturing.ai/topics/networking#ultra-ethernet-specification-published","assertions":[{"id":"73704887-b612-4acd-88ac-022e399a765d","predicate":"event.description","statement":"The Ultra Ethernet Consortium has published a specification at version 1.0.3 for an open AI and HPC networking stack, with AMD, Arista, Broadcom, Cisco, HPE, Intel, Meta, Microsoft and Oracle steering — no release date is established by the source read here.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"46ff712f-9940-47fc-879f-a36048add0b0","stance":"supports","is_primary_for_assertion":false,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"Ultra Ethernet Consortium","publisher":"Ultra Ethernet Consortium","url":"https://ultraethernet.org/","source_type":"other","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]},{"slug":"switch-silicon-passes-100-terabits","title":"A Single Switch Chip Passes 100 Terabits per Second","category":"technology_generation_milestone","event_date":null,"date_precision":"unknown","country":"United States","region":null,"canonical_url":"https://www.manufacturing.ai/topics/networking#switch-silicon-passes-100-terabits","assertions":[{"id":"c6186849-e185-440c-8c70-364ce2757d13","predicate":"event.description","statement":"NVIDIA states 102.4 Tb/s per Spectrum-X switch chip on 200G SerDes with 512 ports of 800G in 5U, against Broadcom's merchant Tomahawk 5 at 51.2 Tb/s — neither source dates its availability, so no date is asserted.","assessment":"reported","editor_reviewed":false,"reference_only_source_count":0,"evidence":[{"id":"ca1fc4a3-8922-4f88-84b2-e1df11eabcfe","stance":"supports","is_primary_for_assertion":true,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"BCM78900 Series (Tomahawk 5) Ethernet Switch","publisher":"Broadcom","url":"https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm78900-series","source_type":"company_website","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}},{"id":"f79ead6c-1a93-4d97-8d43-48250f5ceb5a","stance":"supports","is_primary_for_assertion":true,"origin_independence":"unknown","note":null,"quote":null,"source":{"title":"NVIDIA Spectrum-X Ethernet Networking Platform","publisher":"NVIDIA","url":"https://www.nvidia.com/en-us/networking/spectrumx/","source_type":"company_website","retrieval_status":"fetched","content_inspected":true,"published_at":null,"documents":[]}}]}]}],"policy":{"publication_model":"transparency","evidence_entries_may_share_a_source":true,"linked_documents_do_not_add_verification":true,"correction_intake_available":false,"assessment_meanings":{"primary-source confirmed":"Editor-reviewed against a first-party source.","cross-source corroborated":"Editor-confirmed across two independent origins.","reported":"Supported by the attached sources; not yet editor-reviewed.","disputed":"Sources conflict; competing accounts are both attached.","superseded":"Replaced by a newer assertion.","unverified":"No supporting evidence attached."},"evidence_eligibility":"`evidence[]` contains only sources whose content was actually retrieved and read (`content_inspected: true`). Sources that could not be retrieved are counted per-assertion in `reference_only_source_count`, listed for the whole collection in `consulted_references`, and never presented as evidence. `source_count` counts only sources an assertion actually rests on — it deliberately excludes `consulted_references`, so it is never inflated by material nobody could read.","topic_scope":"Assertions and source counts are limited to the topic's current packet lineage. Reusing a concept in another topic does not silently import that other topic's assertions.","rights":"Statements are Manufacturing.ai's own prose. Source excerpts are not redistributed; `quote` is present only where explicitly cleared.","attribution":"Cite as Manufacturing.ai, https://www.manufacturing.ai/topics/networking","correction_url":"https://www.manufacturing.ai/methodology#corrections"}}