--- title: "The Interconnect Determines How Much of the Chip You Can Use" description: "At rack and pod scale, accelerator utilization depends on scale-up fabrics, scale-out networks, collective software, congestion control, and the physical path carrying every byte." canonical_url: "https://artificialcuriositylabs.ai/posts/2026-08-30-interconnect-matters-more-than-chip/" md_url: "https://artificialcuriositylabs.ai/posts/2026-08-30-interconnect-matters-more-than-chip.md" published_at: "2026-08-30T07:00:00.000Z" tags: - "ai-infrastructure" - "networking" - "interconnect" - "optics" - "data-centers" --- An accelerator can finish its arithmetic and still leave the system waiting. That is the architectural shift behind rack-scale AI. More processors do not automatically produce a faster computer. They produce a larger distributed system whose useful performance depends on moving weights, activations, gradients, expert tokens, and cache state between the right processors at the right time. At rack and pod scale, data movement can constrain useful accelerator performance more than arithmetic. The chip still matters. But the interconnect increasingly decides how much of the chip the system can use. ## TL;DR - Scale-up fabrics connect accelerators into one tightly coupled compute domain; scale-out networks connect servers, racks, and pods. They solve different communication problems. - NVLink and NVSwitch are a mature vertically owned scale-up stack. UALink and Ethernet-based scale-up proposals create open alternatives, but a published specification is not a deployed ecosystem. - InfiniBand and Ethernet/RoCE both support large AI clusters. Their useful performance depends on NIC placement, topology, congestion control, collectives, and operations—not the link rate alone. - NCCL, RCCL, and related libraries turn collective operations into traffic patterns. They are part of the performance architecture, not a thin software wrapper. - Copper remains efficient over short electrical reaches. Pluggable optics dominate longer links. Co-packaged optics and optical I/O move conversion closer to the switch or accelerator as electrical reach and front-panel power become harder constraints. ## Scale-up and scale-out are different systems The cleanest way to understand an AI fabric is to ask where the tightly coupled compute domain ends. **Scale-up** connects accelerators that exchange data frequently enough to behave like one larger machine. Latency, bandwidth, memory semantics, ordering, and collective behavior all matter. The fabric may remain inside a server, cross a rack, or eventually span several racks. **Scale-out** connects those compute domains into a larger cluster. It carries distributed training collectives, inference traffic, model and checkpoint movement, storage access, and control traffic across a broader topology. ```mermaid flowchart LR subgraph R1["Rack-scale compute domain"] A1["Accelerator"] A2["Accelerator"] A3["Accelerator"] SU1["Scale-up switches
NVSwitch, UALink, or another fabric"] A1 <--> SU1 A2 <--> SU1 A3 <--> SU1 end NIC1["NIC or SuperNIC"] SO["Scale-out network
InfiniBand or Ethernet"] NIC2["NIC or SuperNIC"] subgraph R2["Second compute domain"] SU2["Scale-up fabric"] B1["Accelerator"] B2["Accelerator"] SU2 <--> B1 SU2 <--> B2 end SU1 <--> NIC1 NIC1 <--> SO SO <--> NIC2 NIC2 <--> SU2 ``` The distinction is architectural, not merely physical. NVIDIA's [NVLink](https://www.nvidia.com/en-us/data-center/nvlink/) carries tightly coupled GPU traffic, while its [Quantum InfiniBand and Spectrum-X Ethernet platforms](https://www.nvidia.com/en-us/networking/) connect systems across the scale-out network. A cable does not become scale-up because it is fast, and Ethernet does not become scale-out-only because that is its conventional role. That last boundary is now moving. The [UALink Consortium](https://ualinkconsortium.org/specification/) defines an open 200 Gb/s-per-lane scale-up specification for accelerator pods. Broadcom's [Scale-Up Ethernet framework](https://docs.broadcom.com/doc/scale-up-ethernet-framework) applies Ethernet technology to the scale-up domain. AMD's [UALink over Ethernet, or UALoE](https://www.amd.com/en/blogs/2026/amd-helios-resilient-scale-up-networking-for-ai.html), uses that broader direction to connect 72 MI455X GPUs in the Helios rack. The physical and link technologies can converge while the workload contract remains different. ## The scale-up contest is also an ownership contest NVLink's advantage is not only its published bandwidth. NVIDIA owns the GPU interface, NVSwitch silicon, firmware, topology, collective library, rack design, and much of the qualification process. [Blackwell NVLink 5](https://www.nvidia.com/en-us/data-center/nvlink/) provides 1.8 TB/s of aggregate bidirectional bandwidth per GPU. [Rubin NVLink 6](https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/) doubles that published endpoint figure to 3.6 TB/s. In NVL72 systems, NVSwitch extends the tightly coupled domain across 72 GPUs in one rack. Those figures are endpoint totals, not achieved application throughput or one-way fabric bisection bandwidth. Blackwell systems are deployed, while NVIDIA describes Vera Rubin as ramping into full production. Both belong to an integrated product stack—not an interface specification waiting for implementations. UALink takes a different path. Its [200G 1.0 specification](https://ualinkconsortium.org/specification/) separates the fabric definition from any single accelerator vendor and supports scale-up domains as large as 1,024 accelerators. That creates room for accelerators, switch silicon, cables, management components, and systems from different suppliers. The [UALink specification family](https://ualinkconsortium.org/specification/) now also separates common, data-link, physical-layer, chiplet, and manageability documents. The open boundary moves integration responsibility outward. Compatibility requires more than matching electrical lanes: memory semantics, error handling, management, firmware, topology discovery, collectives, and system qualification must converge across companies. | Fabric or initiative | Primary boundary | Ownership model | Status by Aug. 30, 2026 | What the label does not prove | | --------------------------------------------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | | [NVLink + NVSwitch](https://www.nvidia.com/en-us/data-center/nvlink/) | NVIDIA GPU scale-up | Vertically controlled by NVIDIA | Mature across several generations; NVL72 in production | Application bandwidth, availability, or compatibility with non-NVIDIA accelerators | | [UALink specifications](https://ualinkconsortium.org/specification/) | Open accelerator scale-up | Consortium specifications and multi-vendor implementations | 200G 1.0 plus newer component specifications published; implementation ecosystem emerging | Shipping interoperable systems at NVLink maturity | | [Broadcom Scale-Up Ethernet](https://docs.broadcom.com/doc/scale-up-ethernet-framework) | Ethernet-based accelerator scale-up | Merchant switch silicon plus ecosystem | Framework and products published; ecosystem integration under way | That ordinary data-center Ethernet automatically meets scale-up semantics | | [AMD UALoE](https://www.amd.com/en/blogs/2026/amd-helios-resilient-scale-up-networking-for-ai.html) | 72-GPU Helios scale-up domain | AMD implementation using Ethernet switch silicon | Helios architecture introduced in 2026 | Independent interoperability or NVLink-equivalent operational maturity | | [Ultra Ethernet 1.0.3](https://ultraethernet.org/specification-history/) | AI/HPC scale-out transport | Consortium specification | Current specification published; products and software arriving by vendor | A replacement for scale-up fabrics or an installed multi-vendor fleet | These rows are not benchmark peers. NVLink is a deployed proprietary stack. UALink is an open scale-up specification. Ultra Ethernet is a scale-out transport specification. Scale-Up Ethernet is a family of merchant-Ethernet approaches aimed at a new boundary. The strategic question is whether an open ecosystem can coordinate those layers quickly enough to behave like one system. ## Link speed is not collective performance Distributed models do not send a uniform stream of independent packets. They execute collective operations. An **all-reduce** combines values across workers and returns the result to each worker. An **all-gather** assembles shards. A **reduce-scatter** combines and redistributes them. Mixture-of-experts models can produce all-to-all traffic as tokens move toward different experts. [NCCL](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/overview.html) implements these collectives for NVIDIA GPUs across PCIe, NVLink, InfiniBand, and Ethernet paths. [RCCL](https://rocm.docs.amd.com/projects/rccl/en/latest/) provides the corresponding collective library in AMD's ROCm stack. Intel maintains [oneCCL](https://github.com/uxlfoundation/oneCCL) for distributed communication across its supported compute environments. The library chooses algorithms and routes based on topology, message size, available transports, and system configuration. A ring can use links efficiently for large transfers. A tree can reduce latency for other message sizes. Hierarchical algorithms can reduce data inside a server or rack before crossing the scale-out network. ```mermaid flowchart TB OP["Model operation
all-reduce, all-gather, all-to-all"] LIB["Collective library
NCCL, RCCL, or oneCCL"] PLAN["Algorithm and topology choice
ring, tree, hierarchy, channels"] LOCAL["Local movement
HBM, PCIe, scale-up fabric"] NIC["NIC path
RDMA, GPU memory access, transport offloads"] FABRIC["Scale-out fabric
switches, routes, congestion control"] RESULT["Useful synchronized work"] OP --> LIB --> PLAN PLAN --> LOCAL PLAN --> NIC --> FABRIC LOCAL --> RESULT FABRIC --> RESULT ``` This stack explains why a nominally faster network can deliver less useful performance. A job can lose time through a poor rank-to-NIC mapping, oversubscribed links, uneven paths, head-of-line blocking, retransmission, collective imbalance, or one slow worker. The accelerator metric that matters is not peak FLOPS. It is the fraction of execution time spent doing useful compute rather than waiting for communication or synchronization. ## InfiniBand and Ethernet are systems, not cable choices InfiniBand entered AI clusters with an integrated remote-memory and lossless-fabric model. NVIDIA's [Quantum-2 platform](https://www.nvidia.com/en-us/networking/products/infiniband/) combines switches, ConnectX adapters, RDMA, congestion management, telemetry, and its SHARP in-network-reduction capability. In supported Quantum deployments, the fabric can participate in selected collective operations instead of treating every reduction as endpoint-only traffic. Ethernet starts with a broader multi-vendor base. Running AI collectives over Ethernet commonly uses [RoCE](https://docs.nvidia.com/doca/sdk/rdma+over+converged+ethernet/index.html), which carries RDMA over Ethernet. That removes repeated CPU and kernel involvement from the data path, but it does not remove the need to engineer congestion behavior. Traditional RoCE deployments can combine priority flow control, explicit congestion notification, data-center quantized congestion notification, adaptive routing, and careful buffer configuration. NVIDIA's [Spectrum-X](https://www.nvidia.com/en-us/networking/products/ethernet/) packages Ethernet switches, BlueField SuperNICs, telemetry, congestion control, and adaptive routing as one AI-networking platform. The [Ultra Ethernet Consortium's current 1.0.3 specification](https://ultraethernet.org/specification-history/) goes further by defining an AI/HPC transport that includes packet spraying across paths, selective retransmission, congestion management, and collective-communication support. It is an attempt to make large Ethernet fabrics less dependent on every deployment recreating the same behavior through proprietary combinations. | Layer | InfiniBand example | Ethernet example | Operational owner | | ----------------------- | --------------------------------------------- | ------------------------------------------------- | ----------------------------- | | Link and switching | Quantum switches and InfiniBand links | Ethernet switches and links | Network platform team | | Endpoint | ConnectX HCA | NIC or SuperNIC | Server and network teams | | Remote-memory transport | InfiniBand RDMA | RoCE or Ultra Ethernet transport | NIC, driver, and fabric stack | | Congestion behavior | InfiniBand controls and adaptive routing | PFC/ECN/DCQCN, vendor controls, or UEC mechanisms | Fabric engineering | | Collective layer | NCCL/RCCL plus optional in-network operations | NCCL/RCCL/oneCCL plus transport integration | Accelerator platform team | | Job placement | Topology-aware scheduler and rank placement | Topology-aware scheduler and rank placement | Cluster platform team | This is why “InfiniBand versus Ethernet” is an incomplete comparison. The real unit is endpoint hardware, switching, transport, congestion control, collectives, topology, and operations. ## NICs and DPUs own different work A NIC terminates the network path. A modern AI NIC can also expose RDMA, access accelerator memory, steer traffic across rails, timestamp packets, and offload transport work. NVIDIA uses the term **SuperNIC** for a high-performance network endpoint optimized for accelerator traffic. It positions [BlueField-3 SuperNIC](https://www.nvidia.com/en-us/networking/products/data-processing-unit/) around GPU-to-GPU networking, while its DPU configuration adds programmable Arm cores and infrastructure offloads. A **DPU** has a broader ownership boundary. It can run networking, storage, security, isolation, and management functions away from the host CPU. AMD's [Pensando infrastructure products](https://www.amd.com/en/products/data-processing-units/pensando.html) follow that broader offload model. Neither device fixes a poor fabric by itself. A faster NIC attached through the wrong PCIe root, mapped to distant GPUs, or connected to an oversubscribed leaf still creates a slow communication path. Hardware topology and software placement have to agree. ## Copper reaches a wall before optics does The physical medium is becoming an architectural choice. Passive copper is attractive inside a tray or across short rack distances because it avoids optical conversion and can deliver low power and low latency. Active electrical cables extend that reach by adding signal conditioning. As lane rates rise, electrical loss increases and the practical reach shrinks. Pluggable optical modules move the electrical-to-optical conversion to the switch faceplate. They provide longer reach and replaceable modules, but every port consumes front-panel area and conversion power. Linear pluggable optics remove part of the digital signal processing from the module to reduce power, while placing tighter signal-integrity requirements on the host system. Co-packaged optics moves the optical engines beside the switch ASIC. Broadcom's [co-packaged-optics portfolio](https://www.broadcom.com/products/fiber-optic-modules-components/co-packaged-optics) and NVIDIA's [silicon-photonics switch platforms](https://www.nvidia.com/en-us/networking/products/silicon-photonics/) shorten the high-speed electrical path and increase optical bandwidth density. They also couple optics more closely to switch packaging, cooling, yield, and service procedures. ```mermaid flowchart TB NEED["Required reach, bandwidth density,
power, and service model"] TRACE["On-board traces
inside board or package"] COPPER["Passive or active copper
tray and short rack links"] PLUG["Pluggable optics
conversion at faceplate"] CPO["Co-packaged optics
conversion beside switch ASIC"] OIO["Optical I/O
conversion beside compute or memory"] NEED --> TRACE NEED --> COPPER NEED --> PLUG NEED --> CPO NEED --> OIO ``` Optical I/O pushes the boundary closer to compute. [Ayar Labs' TeraPHY](https://ayarlabs.com/products/) packages optical chiplets for chip-to-chip communication. [Lightmatter Passage](https://lightmatter.co/products/passage/) combines optical connectivity with packaging intended to link processors and memory. [Celestial AI's Photonic Fabric](https://www.celestial.ai/technology) targets optical scale-up and memory connectivity. These products do not share one maturity level or deployment model: | Optical boundary | Public maturity by Aug. 30, 2026 | Evidence boundary | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | ----------------------------------------------------------------------- | | [Pluggable optics](https://www.nvidia.com/en-us/networking/interconnect/) | Shipping infrastructure across data-center networks | Mature module ecosystem; power and faceplate density remain constraints | | [Broadcom CPO](https://www.broadcom.com/products/fiber-optic-modules-components/co-packaged-optics) and [NVIDIA silicon photonics](https://www.nvidia.com/en-us/networking/products/silicon-photonics/) | Commercial portfolios and announced switch platforms | Product availability varies by platform and configuration | | [Ayar Labs optical I/O](https://ayarlabs.com/products/) | Chiplet products and customer sampling programs | Not yet a broadly interchangeable accelerator interface | | [Lightmatter Passage](https://lightmatter.co/products/passage/) and [Celestial AI Photonic Fabric](https://www.celestial.ai/technology) | Product programs and partner integrations | Deployment scale and production economics remain vendor-reported | The direction is consistent even though the maturity is not: when electrical reach, connector density, and SerDes power become system limits, designers can move the optical conversion point inward. That choice is driven by distance and packaging boundary, not one universal copper-to-optics migration sequence. ## The interconnect changes who owns the computer NVIDIA's model places the scale-up link, switch, NIC, network, collective library, and rack qualification under one platform owner. That reduces the number of boundaries a system integrator must reconcile. An open stack distributes those responsibilities. An accelerator vendor can implement UALink. A merchant-silicon supplier can provide scale-up and scale-out switches. A NIC vendor can own RDMA and congestion functions. An OEM can assemble the rack. A collective library can adapt the topology to the framework. That structure creates supplier choice. It also creates a harder acceptance test. A specification match does not establish that the assembled system sustains collectives under congestion, survives link failures, maps jobs correctly, or exposes one support boundary when performance falls. The practical evaluation unit is therefore not the accelerator card. It is: 1. the tightly coupled scale-up domain; 2. the scale-out fabric joining those domains; 3. the collective software translating model operations into traffic; 4. the topology and congestion mechanisms preserving progress; 5. the physical medium carrying the required bandwidth within the power and service envelope. ## What remains unproven Vendor bandwidth figures are abundant. Comparable useful-work evidence is not. A credible system comparison needs the same model, precision, collective mix, accelerator count, topology, message sizes, failure policy, and power boundary. It also needs achieved bandwidth, communication-to-compute overlap, tail behavior under congestion, job completion time, and recovery behavior—not only a clean microbenchmark. Open fabrics face a second proof burden: interoperability. UALink and Ultra Ethernet can publish complete specifications before multiple vendors ship independently qualified systems. That is normal standards work, but it creates a gap between protocol maturity and operational maturity. ## So what The next AI platform decision cannot stop at chip throughput. The fabric determines the size of the useful computer, the software determines how traffic enters it, and optics determines how far that computer can extend before power and signal integrity pull it apart. The open question is whether the industry can create a genuinely interchangeable scale-up and scale-out ecosystem—or whether every high-performing AI cluster will remain a platform-specific computer whose compatibility ends exactly where its collective software, congestion controls, and optical packaging diverge. --- _Part of the [AI Compute Landscape](https://artificialcuriositylabs.ai/series/ai-compute/) — an ongoing exploration of accelerator architectures, software stacks, and data-center systems._