Why AI’s Next Bottleneck May Be Moving Data, Not Processing It

Why AI’s Next Bottleneck May Be Moving Data, Not Processing It

AI is becoming a distributed-computing problem. HBM feeds individual accelerators, but scale-up and scale-out fabrics determine whether dozens, thousands, and eventually far larger numbers of accelerators can synchronize without wasting expensive compute cycles. Rapid growth in AI networking, retimers, active electrical cables, optical DSPs, switches and silicon photonics suggests that data movement is becoming a material source of both bottleneck risk and semiconductor value creation.

Executive Summary

The AI infrastructure discussion still tends to start with accelerator counts. That is increasingly incomplete. A high-value accelerator creates economic value only when it spends time computing rather than waiting for memory, synchronization, or another processor. As clusters scale, the network itself becomes part of the machine. NVIDIA’s sixth-generation NVLink delivers 3.6 TB/s of bidirectional GPU-to-GPU bandwidth per Rubin GPU and 260 TB/s across a 72-GPU NVL72 domain [1]. UALink 1.0, the emerging open alternative for scale-up fabrics, specifies 200 Gb/s per lane and domains of up to 1,024 accelerators [2].

This article focuses on a different constraint than MEMSWork’s earlier HBM and advanced-packaging analyses. Those articles examined how accelerators are supplied and assembled; this one examines what happens after the accelerators exist: how efficiently they exchange data across a rack, cluster and data-center campus. The investment question is therefore not simply who builds faster processors, but who supplies the connective tissue that keeps those processors utilized.

Key Findings

1. The Bottleneck Is Moving From the Chip to the Fabric

The physical reason is simple: modern AI workloads are highly parallel but not independent. Training requires collective operations such as all-reduce and broadcast; mixture-of-experts inference creates bursts of all-to-all traffic; model and context state must move among accelerators. When a processor waits for another device, its theoretical FLOPS do not generate tokens or revenue. The relevant metric is therefore closer to productive accelerator-hours than installed accelerator-hours.

NVIDIA’s roadmap illustrates how aggressively the industry is attacking this problem. NVLink 6 provides 3.6 TB/s per GPU and 260 TB/s of aggregate rack-level GPU bandwidth in a 72-GPU Vera Rubin NVL72 domain. NVIDIA also describes future scale-up domains reaching 1,152 GPUs [1]. In parallel, UALink 1.0 defines an open 200 Gb/s-per-lane fabric for as many as 1,024 accelerators [2]. The important strategic question is whether scale-up remains vertically integrated with accelerator platforms or becomes a broader merchant-semiconductor market.

Figure 1. The AI data-movement hierarchy. Scale-up links accelerators within a tightly coupled domain; scale-out connects racks and pods; scale-across links larger facilities. Source: MEMSWork synthesis based on NVIDIA, UALink, UEC and Broadcom roadmaps [1-4].

2. Scale-Out Networking Is Becoming Part of Compute Economics

Beyond the rack, networking must preserve utilization across much larger clusters. NVIDIA’s ConnectX-9 delivers up to 800 Gb/s per port and up to 1.6 Tb/s of throughput to each Rubin GPU [3]. Broadcom’s Tomahawk 6, now shipping in production volume, provides 102.4 Tb/s of switching capacity in one Ethernet switch ASIC [4]. Arista introduced 1.6 Tb/s AI-fabric platforms in 2026 as customers expanded scale-up, scale-out and scale-across networks [5]. Ultra Ethernet Consortium specification 1.0.3, released in July 2026, formalizes an Ethernet architecture specifically tuned for AI and HPC traffic [6].

These technologies address more than headline bandwidth. AI traffic is synchronized and bursty, so tail latency, congestion control, packet scheduling and link stability can determine the useful throughput of the entire cluster. A slow or congested fabric can strand compute capacity that has already been paid for. This is why in-network compute, smarter endpoint scheduling and AI-specific congestion control are becoming strategic features rather than networking details.

Figure 2. Bandwidth requirements expand across the AI fabric. Metrics refer to different scopes and are not directly comparable; the chart illustrates how capacity requirements escalate from endpoints to switch systems. Sources: NVIDIA and Broadcom [1,3,4,7].

3. Corporate Results Suggest Value Is Already Moving Into Connectivity

The strongest investor evidence is financial rather than architectural. Broadcom reported fiscal Q3 2026 AI semiconductor revenue of $16.7 billion, up 221% year over year, driven by custom AI accelerators and networking; it guided Q4 AI semiconductor revenue to $21.7 billion [8]. Astera Labs reported Q2 revenue of $392.4 million, up 104%, and said its Scorpio X-Series fabric switches were on track to become its largest product family [9]. Credo reported fiscal Q1 2027 revenue of $479.0 million, up 114.7% year over year, across copper and optical connectivity solutions [10].

The trend extends to more mature suppliers. Marvell’s fiscal Q2 2027 revenue rose 37% to $2.739 billion while Data Center revenue grew 46%; the company separately states that optical interconnect has compounded at roughly 50% annually for five years and now represents about half of its Data Center revenue [11,12]. Arista’s Q2 2026 revenue reached $3.036 billion, up 37.7% year over year [5]. These metrics cover different businesses and should not be treated as directly comparable, but the direction is consistent: the connective layer around the accelerator is becoming economically material.

Figure 3. Selected connectivity-related growth metrics. Company-specific metrics differ in scope and are not directly comparable. Sources: Broadcom, Credo, Astera Labs, Marvell and Arista company disclosures [5,8-11].

4. The Electrical Wall Is Pulling Optics Closer to the Processor

The next bottleneck is physical. Higher electrical signaling rates require more power for equalization and retiming, while loss rises with distance and channel complexity. That is expanding the role of active electrical cables, retimers and DSPs in the near term, while pushing optics progressively closer to the switch ASIC over the longer term.

NVIDIA’s Spectrum-X Photonics platform integrates silicon-photonics engines through co-packaged optics and offers up to 409.6 Tb/s of switch-system bandwidth. NVIDIA claims the CPO architecture can provide about five times better network power efficiency than traditional pluggable transceivers; this is a vendor engineering claim, not an industry-wide guarantee [7]. Broadcom is following the same direction with its third-generation CPO portfolio and 200 Gb/s-per-lane optical engines, while its October 2026 OCP roadmap spans Tomahawk 6, Tomahawk Ultra and Jericho 4 across scale-up, scale-out and scale-across networks [13].

5. How the Data-Movement Bottleneck Gets Relieved

Table 1. Data-movement constraints, likely paths to relief, and enabling innovation opportunities. Source: MEMSWork synthesis based on company and consortium materials cited below.

The solution is therefore layered rather than singular. Open scale-up standards can broaden the supplier base. Smarter Ethernet can reduce congestion and preserve effective bandwidth. Retimers and active electrical cables can extend copper where it remains economical. Co-packaged optics and silicon photonics can lower the energy cost of moving bits over longer distances. Software matters as much as line rate: topology-aware collective libraries, network telemetry and workload scheduling can increase useful GPU utilization without adding more accelerators.

For investors, the opportunity set is broader than the largest switch vendors. The enabling layer includes high-speed SerDes, PCIe/CXL and UALink switching, retimers, active electrical cables, optical DSPs, silicon-photonics engines, lasers, fiber connectors, photonics packaging and test, as well as congestion-control and collective-communication software. The common economic objective is to reduce the cost and energy required to move each useful bit while keeping expensive accelerators busy.

Investor Takeaway

The article’s conclusion is not that networking has replaced compute as the AI bottleneck. It is that AI is becoming a distributed-computing system in which data movement increasingly determines how much value installed compute can produce. A faster GPU is valuable only if memory and network fabrics can feed and synchronize it.

The indicators worth tracking are therefore expanding: scale-up domain size, bandwidth per accelerator, switch radix and throughput, optical-interconnect revenue, retimer and AEC adoption, CPO qualification, fabric utilization and customer spending on AI networking. If these technologies reduce communication overhead, the payoff is not merely faster networks; it is higher utilization of the entire AI capital base.

The strategic question is shifting from how many accelerators can be deployed to how efficiently thousands of accelerators can behave as one machine. Companies that solve that problem may capture an increasing share of AI infrastructure value even if they never manufacture the accelerator itself.

References

[1] NVIDIA, “NVIDIA NVLink: The Scale-Up Network for AI Factories,” July 20, 2026. Link

[2] UALink Consortium, UALink 200G 1.0 Specification. Link

[3] NVIDIA, ConnectX-9 SuperNIC / Vera Rubin networking specifications, 2026. Link

[4] Broadcom, “Tomahawk 6 Now Shipping in Production Volume,” March 12, 2026. Link

[5] Arista Networks, Q2 2026 Financial Results, August 4, 2026. Link

[6] Ultra Ethernet Consortium, Specification History; v1.0.3 released July 16, 2026. Link

[7] NVIDIA, Spectrum-X Ethernet Photonics / Silicon Photonics, 2026. Link

[8] Broadcom, Q3 FY2026 Financial Results, September 2026. Link

[9] Astera Labs, Q2 2026 Financial Results, August 4, 2026. Link

[10] Credo Technology, Q1 FY2027 Financial Results, September 1, 2026. Link

[11] Marvell Technology, Q2 FY2027 Financial Results, August 27, 2026. Link

[12] Marvell Technology, 2026 Proxy Statement: optical-interconnect growth and data-center mix. Link

[13] Broadcom, OCP 2026 AI Networking Innovations, October 8, 2026. Link

Research note: All numerical statements are sourced to primary company or consortium materials where available. Company growth metrics differ in scope and are labeled accordingly. Vendor performance claims, including co-packaged-optics efficiency claims, are identified as vendor claims rather than universal benchmarks.