How do UCIe and BoW interconnects support generative AI on chiplets?

The bunch-of-wires (BoW) and Universal Chiplet Interconnect Express (UCIe) standards provide designers with tradeoffs in terms of throughput, interconnect density, delay, and bump pitch. This FAQ compares the performance of BoW and UCIe and looks at how optical interconnects may provide a path to even higher performance interconnects in chiplets.

To realize optimal performance for generative artificial intelligence (AI), machine learning (ML), and other high-performance computing (HPC) applications, designers are turning to chipsets that can combine AI accelerators, GPUs, CPU, memory, and networking in a single package. Interconnecting heterogeneous devices in a chiplet can be especially challenging.

BoW was developed by the Open Domain Specific Architecture (ODSA) working group within the Open Compute Project (OCP) organization. The goals of BoW are to provide low implementation costs, compatibility with different IC process nodes, low power, low latency, and the ability to support a variety of packaging technologies used to assemble chipsets, like low-cost laminates and high-performance silicon-based interposers.

BoW 2.0 is the most recent version and doubled the top speed to 512 Gigabits per second (Gbps) per 16-lane slice. The updated standard also improved energy efficiency with gated clock and data line inactive modes that can save up to 90% of power compared to alternative approaches, half-width slices for smaller dies with limited space for interconnects, channel management using sideband slices, direction configuration across channels with bidirectional slices and bump redundancy to improve robustness of the interconnects.

In one case, a digital in-memory-compute (DIMC) chiplet has been developed using the BoW PHY standard. The low latency of BoW supports AI inference using large language models (LLMs). The new chipset solution delivers a 40x improvement in memory bandwidth compared to high-performance GPUs. The higher memory bandwidth produces higher throughput and lower latency for generative inference applications while minimizing total costs.

As shown, BoW supports high-performance interconnects for generative AI and other HPC applications. In instances where even more performance is required, designers can consider UCIe.

How UCIe compares
The UCIe specifications include the die-to-die (D2D) I/O physical layer, D2D protocols, and software stack. UCIe leverages the PCI Express (PCIe) and Compute Express Link (CXL) industry standards. Like the latest version of BoW, UCIe 1.1 includes redundancy to support high reliability. In addition, UCIe 1.1 has expanded support for AI in autonomous vehicles. Some other changes in UCIe 1.1 include:

Architectural enhancements to improve compliance testing.
Supports simultaneous multiprotocol with full link layer functionality for streaming protocols.
Runtime health monitoring and repair for automotive and high-reliability applications.
New bump maps support lower-cost packaging.

UCIe supports slightly higher performance compared with BoW. UCIe and BoW are generally comparable, but UCIe has been optimized for higher-density applications (Table 1).

Table 1. UCIe and BoW offer similar levels of performance, but UCIe can support higher-density applications (Table: *Cadence*).

While BoW is designed primarily for use inside chiplets, UCIe is designed for two use cases. Like BoW, UCIe is designed to support power-efficient, high-density, and cost-effective interconnect inside chipsets. In addition, UCIe is designed to support off-package connectivity using optical and electrical interconnects using retimers for the underlying protocol like PCIe or CXL at the rack level and higher.

When using UCIe for connectivity outside a chiplet, security can become a concern. In response, the UCIe consortium has established a sub-working group focused on enhanced security for chiplet and multi-die ecosystems. This effort will address the multiple security challenges internal to chiplets and for connecting chiplets to the outside world. A secure AI-empowered core protocol layer has been proposed to address the growing security challenges (Figure 1). As chipsets find expanding use in automotive systems, AI, ML, and HPC applications, the need for security is expected to grow.

Figure 1. UCIe can include a secure AI-empowered protocol layer to address security needs (Image: *UCIexpress.org*).

Summary
BoW and UCIe both support high levels of performance and redundancy for chiplet interconnects. UCIe can deliver higher-density solutions. BoW is primarily designed for use inside a chiplet, while UCIe can support communications inside chiplets and between chiplets and outside systems.

References
Bunch of Wires (BoW) PHY Specification, Open Compute Project
Design and technology spaces for heterogeneous chiplet integration, Design the Solution
Generative AI Compute Front-Runner d-Matrix Reaches New Milestone in Efficient AI Inference, d-Matrix
Securing the New Frontier: Chiplets & Hardware Security Challenges, UCIexpress.org
The Bunch of Wires (BoW) – An Open-Source Physical Interface Enabling Chiplet Architectures, IEEE Electronics Packaging Society
UCIe Standard: Benefits & Requirements Explained, Synopsys
Universal Chiplet Interconnect Express (UCIe)®: An open standard for developing a successful chiplet ecosystem, IEEE Electronics Packaging Society
Which Data Interfaces Are Used for Chiplet Interconnects?, Cadence