The Interconnect Determines How Much of the Chip You Can Use

An accelerator can finish its arithmetic and still leave the system waiting.

That is the architectural shift behind rack-scale AI. More processors do not automatically produce a faster computer. They produce a larger distributed system whose useful performance depends on moving weights, activations, gradients, expert tokens, and cache state between the right processors at the right time.

At rack and pod scale, data movement can constrain useful accelerator performance more than arithmetic. The chip still matters. But the interconnect increasingly decides how much of the chip the system can use.

TL;DR

• Scale-up fabrics connect accelerators into one tightly coupled compute domain; scale-out networks connect servers, racks, and pods. They solve different communication problems.
• NVLink and NVSwitch are a mature vertically owned scale-up stack. UALink and Ethernet-based scale-up proposals create open alternatives, but a published specification is not a deployed ecosystem.
• InfiniBand and Ethernet/RoCE both support large AI clusters. Their useful performance depends on NIC placement, topology, congestion control, collectives, and operations—not the link rate alone.
• NCCL, RCCL, and related libraries turn collective operations into traffic patterns. They are part of the performance architecture, not a thin software wrapper.
• Copper remains efficient over short electrical reaches. Pluggable optics dominate longer links. Co-packaged optics and optical I/O move conversion closer to the switch or accelerator as electrical reach and front-panel power become harder constraints.

Scale-up and scale-out are different systems

The cleanest way to understand an AI fabric is to ask where the tightly coupled compute domain ends.

Scale-up connects accelerators that exchange data frequently enough to behave like one larger machine. Latency, bandwidth, memory semantics, ordering, and collective behavior all matter. The fabric may remain inside a server, cross a rack, or eventually span several racks.

Scale-out connects those compute domains into a larger cluster. It carries distributed training collectives, inference traffic, model and checkpoint movement, storage access, and control traffic across a broader topology.

The distinction is architectural, not merely physical. NVIDIA’s NVLink carries tightly coupled GPU traffic, while its Quantum InfiniBand and Spectrum-X Ethernet platforms connect systems across the scale-out network. A cable does not become scale-up because it is fast, and Ethernet does not become scale-out-only because that is its conventional role.

That last boundary is now moving. The UALink Consortium defines an open 200 Gb/s-per-lane scale-up specification for accelerator pods. Broadcom’s Scale-Up Ethernet framework applies Ethernet technology to the scale-up domain. AMD’s UALink over Ethernet, or UALoE, uses that broader direction to connect 72 MI455X GPUs in the Helios rack. The physical and link technologies can converge while the workload contract remains different.

The scale-up contest is also an ownership contest

NVLink’s advantage is not only its published bandwidth. NVIDIA owns the GPU interface, NVSwitch silicon, firmware, topology, collective library, rack design, and much of the qualification process.

Blackwell NVLink 5 provides 1.8 TB/s of aggregate bidirectional bandwidth per GPU. Rubin NVLink 6 doubles that published endpoint figure to 3.6 TB/s. In NVL72 systems, NVSwitch extends the tightly coupled domain across 72 GPUs in one rack.

Those figures are endpoint totals, not achieved application throughput or one-way fabric bisection bandwidth. Blackwell systems are deployed, while NVIDIA describes Vera Rubin as ramping into full production. Both belong to an integrated product stack—not an interface specification waiting for implementations.

UALink takes a different path. Its 200G 1.0 specification separates the fabric definition from any single accelerator vendor and supports scale-up domains as large as 1,024 accelerators. That creates room for accelerators, switch silicon, cables, management components, and systems from different suppliers.

The UALink specification family now also separates common, data-link, physical-layer, chiplet, and manageability documents. The open boundary moves integration responsibility outward. Compatibility requires more than matching electrical lanes: memory semantics, error handling, management, firmware, topology discovery, collectives, and system qualification must converge across companies.

These rows are not benchmark peers. NVLink is a deployed proprietary stack. UALink is an open scale-up specification. Ultra Ethernet is a scale-out transport specification. Scale-Up Ethernet is a family of merchant-Ethernet approaches aimed at a new boundary.

The strategic question is whether an open ecosystem can coordinate those layers quickly enough to behave like one system.

Link speed is not collective performance

Distributed models do not send a uniform stream of independent packets. They execute collective operations.

An all-reduce combines values across workers and returns the result to each worker. An all-gather assembles shards. A reduce-scatter combines and redistributes them. Mixture-of-experts models can produce all-to-all traffic as tokens move toward different experts.

NCCL implements these collectives for NVIDIA GPUs across PCIe, NVLink, InfiniBand, and Ethernet paths. RCCL provides the corresponding collective library in AMD’s ROCm stack. Intel maintains oneCCL for distributed communication across its supported compute environments.

The library chooses algorithms and routes based on topology, message size, available transports, and system configuration. A ring can use links efficiently for large transfers. A tree can reduce latency for other message sizes. Hierarchical algorithms can reduce data inside a server or rack before crossing the scale-out network.

This stack explains why a nominally faster network can deliver less useful performance. A job can lose time through a poor rank-to-NIC mapping, oversubscribed links, uneven paths, head-of-line blocking, retransmission, collective imbalance, or one slow worker.

The accelerator metric that matters is not peak FLOPS. It is the fraction of execution time spent doing useful compute rather than waiting for communication or synchronization.

InfiniBand and Ethernet are systems, not cable choices

InfiniBand entered AI clusters with an integrated remote-memory and lossless-fabric model. NVIDIA’s Quantum-2 platform combines switches, ConnectX adapters, RDMA, congestion management, telemetry, and its SHARP in-network-reduction capability. In supported Quantum deployments, the fabric can participate in selected collective operations instead of treating every reduction as endpoint-only traffic.

Ethernet starts with a broader multi-vendor base. Running AI collectives over Ethernet commonly uses RoCE, which carries RDMA over Ethernet. That removes repeated CPU and kernel involvement from the data path, but it does not remove the need to engineer congestion behavior.

Traditional RoCE deployments can combine priority flow control, explicit congestion notification, data-center quantized congestion notification, adaptive routing, and careful buffer configuration. NVIDIA’s Spectrum-X packages Ethernet switches, BlueField SuperNICs, telemetry, congestion control, and adaptive routing as one AI-networking platform.

The Ultra Ethernet Consortium’s current 1.0.3 specification goes further by defining an AI/HPC transport that includes packet spraying across paths, selective retransmission, congestion management, and collective-communication support. It is an attempt to make large Ethernet fabrics less dependent on every deployment recreating the same behavior through proprietary combinations.

This is why “InfiniBand versus Ethernet” is an incomplete comparison. The real unit is endpoint hardware, switching, transport, congestion control, collectives, topology, and operations.

NICs and DPUs own different work

A NIC terminates the network path. A modern AI NIC can also expose RDMA, access accelerator memory, steer traffic across rails, timestamp packets, and offload transport work.

NVIDIA uses the term SuperNIC for a high-performance network endpoint optimized for accelerator traffic. It positions BlueField-3 SuperNIC around GPU-to-GPU networking, while its DPU configuration adds programmable Arm cores and infrastructure offloads.

A DPU has a broader ownership boundary. It can run networking, storage, security, isolation, and management functions away from the host CPU. AMD’s Pensando infrastructure products follow that broader offload model.

Neither device fixes a poor fabric by itself. A faster NIC attached through the wrong PCIe root, mapped to distant GPUs, or connected to an oversubscribed leaf still creates a slow communication path. Hardware topology and software placement have to agree.

Copper reaches a wall before optics does

The physical medium is becoming an architectural choice.

Passive copper is attractive inside a tray or across short rack distances because it avoids optical conversion and can deliver low power and low latency. Active electrical cables extend that reach by adding signal conditioning. As lane rates rise, electrical loss increases and the practical reach shrinks.

Pluggable optical modules move the electrical-to-optical conversion to the switch faceplate. They provide longer reach and replaceable modules, but every port consumes front-panel area and conversion power. Linear pluggable optics remove part of the digital signal processing from the module to reduce power, while placing tighter signal-integrity requirements on the host system.

Co-packaged optics moves the optical engines beside the switch ASIC. Broadcom’s co-packaged-optics portfolio and NVIDIA’s silicon-photonics switch platforms shorten the high-speed electrical path and increase optical bandwidth density. They also couple optics more closely to switch packaging, cooling, yield, and service procedures.

Optical I/O pushes the boundary closer to compute. Ayar Labs’ TeraPHY packages optical chiplets for chip-to-chip communication. Lightmatter Passage combines optical connectivity with packaging intended to link processors and memory. Celestial AI’s Photonic Fabric targets optical scale-up and memory connectivity.

These products do not share one maturity level or deployment model:

The direction is consistent even though the maturity is not: when electrical reach, connector density, and SerDes power become system limits, designers can move the optical conversion point inward. That choice is driven by distance and packaging boundary, not one universal copper-to-optics migration sequence.

The interconnect changes who owns the computer

NVIDIA’s model places the scale-up link, switch, NIC, network, collective library, and rack qualification under one platform owner. That reduces the number of boundaries a system integrator must reconcile.

An open stack distributes those responsibilities. An accelerator vendor can implement UALink. A merchant-silicon supplier can provide scale-up and scale-out switches. A NIC vendor can own RDMA and congestion functions. An OEM can assemble the rack. A collective library can adapt the topology to the framework.

That structure creates supplier choice. It also creates a harder acceptance test. A specification match does not establish that the assembled system sustains collectives under congestion, survives link failures, maps jobs correctly, or exposes one support boundary when performance falls.

The practical evaluation unit is therefore not the accelerator card. It is:

1. the tightly coupled scale-up domain;
2. the scale-out fabric joining those domains;
3. the collective software translating model operations into traffic;
4. the topology and congestion mechanisms preserving progress;
5. the physical medium carrying the required bandwidth within the power and service envelope.

What remains unproven

Vendor bandwidth figures are abundant. Comparable useful-work evidence is not.

A credible system comparison needs the same model, precision, collective mix, accelerator count, topology, message sizes, failure policy, and power boundary. It also needs achieved bandwidth, communication-to-compute overlap, tail behavior under congestion, job completion time, and recovery behavior—not only a clean microbenchmark.

Open fabrics face a second proof burden: interoperability. UALink and Ultra Ethernet can publish complete specifications before multiple vendors ship independently qualified systems. That is normal standards work, but it creates a gap between protocol maturity and operational maturity.

So what

The next AI platform decision cannot stop at chip throughput. The fabric determines the size of the useful computer, the software determines how traffic enters it, and optics determines how far that computer can extend before power and signal integrity pull it apart.

The open question is whether the industry can create a genuinely interchangeable scale-up and scale-out ecosystem—or whether every high-performing AI cluster will remain a platform-specific computer whose compatibility ends exactly where its collective software, congestion controls, and optical packaging diverge.

Part of the AI Compute Landscape — an ongoing exploration of accelerator architectures, software stacks, and data-center systems.