100 GbE RDMA: High-Speed Memory Access

Updated 19 February 2026

100 GbE RDMA is a high-speed Ethernet protocol architecture enabling remote direct memory access with zero-copy transfers, kernel bypass, and in-network compute.
It employs standards like RoCE v2 and custom minimal header approaches on ASIC, FPGA, and SmartNIC platforms to achieve near-wire throughput and microsecond-scale latencies.
Applications span distributed machine learning, high-performance computing, and memory disaggregation, demonstrating significant improvements in throughput, latency, and system scaling.

100 Gigabit Ethernet Remote Direct Memory Access (100 GbE RDMA) comprises a class of hardware and protocol architectures that deliver direct, high-bandwidth, low-latency memory access semantics across 100 Gb/s Ethernet. These solutions eschew conventional host-bound data movement by enabling zero-copy transfers, kernel bypass, and—in recent SmartNIC and programmable data center deployments—direct in-network compute and offloaded memory semantics. Modern 100 GbE RDMA implementations span industry standards such as RoCE v2, customized FPGA-based protocols, and open stacks supporting SmartNIC-accelerated line-rate processing, with broad application in distributed machine learning, data acquisition, high-performance computing, and memory disaggregation.

1. Protocol Architectures and Semantics

100 GbE RDMA protocols build on the Ethernet PHY and MAC layers while layering transport semantics to support remote memory operations. RoCE v2 (RDMA over Converged Ethernet v2) is the prevalent industry standard, encapsulating InfiniBand transport headers in UDP/IP over Ethernet. A distinctive protocol property is the zero-copy, direct host memory access through registered RDMA memory regions, managed by the NIC Queue Pair (QP) state machine. Standard RoCE v2 packets consist of a 66 B header (inclusive of IB-GRH and IB-BTH), which, while supporting rich transport features (retransmission, credits, completion semantics), imposes per-packet overhead that becomes pronounced at high packet rates (e.g., >1.4 Gpps for small 8 B payloads at 100 Gb/s) (Hoefler et al., 2023).

Custom 100 GbE RDMA proposals eliminate excess header fields—replacing InfiniBand-specific headers and CRCs with lightweight UDP or custom control fields—to maximize payload efficiency and achievable packet rates (Mansour et al., 2018). Emerging approaches extend semantics beyond traditional read, write, and send operations to support direct in-network memory pooling and compute via programmable instruction sets (e.g., NetDAM’s in-payload opcode dispatch mechanism) (Fang et al., 2021) and native memory disaggregation (e.g., CXL-load/store over 100 GbE) (Wang et al., 2023).

Programmability is a central axis of differentiation: while ASIC-based adapters implement fixed RDMA logic, FPGA- and SmartNIC-based solutions (e.g., RoCE BALBOA, NetDAM) expose the datapath for user-defined operations, encryption, or line-rate analytics (Heer et al., 27 Jul 2025, Fang et al., 2021).

2. Hardware Implementations and Data Paths

ASIC NICs (Mellanox, Broadcom, Intel) deliver 100 GbE RoCE via tightly integrated switching and transport engines, with host-side integration through PCIe and direct memory access. These NICs handle protocol offload (header parsing, retransmission, flow/congestion control, completion queues) at fixed pipeline depths, with host CPUs coordinating resource management and QP allocation.

FPGA and SmartNIC-based architectures utilize the programmable logic fabric for custom protocol processing, memory attachment, and in-network function offload. NetDAM, as an exemplar, is implemented on Xilinx Alveo U55N FPGA cards, directly coupling DRAM (2 GB HBM in prototype) to the 100 GbE MAC/PCS via a packet steering, parsing, and programmable ISA engine; programmable SIMD ALUs perform in-flight computation, supporting up to ~2048 float32 ops/cycle at 250 MHz (raw compute: 512 Gb/s) (Fang et al., 2021). Similarly, RoCE BALBOA leverages AMD Alveo U55C with HBM-backed buffering and a fully pipelined packet engine capable of supporting hundreds of QPs and line-rate operation (Heer et al., 27 Jul 2025).

All architectures incorporate high-throughput DMA engines (XDMA/QDMA), often supporting peer-to-peer transfers (e.g., direct DPU→GPU RDMA in BALBOA), and on-chip queue management for request/completion notification. Modern systems support programmable datapath slots for in-place processing (ML-based DPI, cryptography, pre-processing pipelines) and dynamic adaptation via user-space verbs (Heer et al., 27 Jul 2025).

CXL-over-Ethernet innovations extend 100 GbE RDMA to CPU-memory decoupled, disaggregated systems, encapsulating native CXL flits into custom 89 B Ethernet frames handled entirely in FPGA logic, preserving cache-coherent load/store semantics (Wang et al., 2023).

3. Flow Control, Reliability, and Congestion Management

Legacy 100 GbE RDMA deployments rely on Priority Flow Control (PFC) to provide lossless transport, as go-back-N retransmission amplifies any single drop into large-scale bandwidth waste and head-of-line blocking; at 100 Gb/s and RTT = 3 µs, a single loss can retrigger ≈375 KiB of re-sent data (Hoefler et al., 2023). However, PFC introduces risks such as congestion spreading and deadlocks.

Recent work (IRN: Improved RoCE NIC) demonstrates that selective retransmission (SACK) plus bandwidth-delay-product (BDP) flow control can supplant PFC. IRN’s approach (i) uses receiver-generated lightweight NACKs per out-of-order packet, (ii) sender-side SACK bitmaps, and (iii) BDP-based in-flight window caps, yielding 2×–4× lower tail latency, 30–70% higher throughput under congestion, and allowing PFC to be safely disabled in all-IRN networks (Mittal et al., 2018, Hoefler et al., 2023). These mechanisms add 3–10% NIC fabric cost and require minor packet header extensions. Modern protocol stacks incorporate ECN-based congestion control (DCQCN, TIMELY, HPCC) and, increasingly, hop-by-hop backpressure techniques (BPFC) for fine-grained flow management (Hoefler et al., 2023).

Packet loss detection is optimized using rolling-window shift registers for lightweight acknowledgment (detecting missing packets within 512-packet windows) (Mansour et al., 2018), and FEC schemes (RS(544, 514)) provide forward error correction at minimal latency (~70–150 ns/block) (Hoefler et al., 2023). Selective re-transmit engines reduce recovery latency by ≈35% over go-back-N (Wang et al., 2023).

4. Performance: Throughput, Latency, and Resource Metrics

100 GbE RDMA architectures consistently approach near-wire throughput for payloads ≥32 KB (e.g., sustained 90–98 Gb/s on FPGA→FPGA links, 95.6 Gb/s in streaming data acquisition, 100 Gb/s in RoCE BALBOA), with larger payload and batch sizes amortizing per-packet overheads (Mansour et al., 2018, Fang et al., 2021, Villani et al., 26 Jan 2026, Heer et al., 27 Jul 2025). ASIC and FPGA NICs matching link rate in RDMA WRITE. Small-packet performance favors custom RDMA designs (≥50% higher bandwidth at 598 B packets; minimal headers/out-of-order support), as RoCE v2 header and iCRC overheads depress effective Gpps rates (Mansour et al., 2018, Hoefler et al., 2023).

Microsecond-scale latency is characteristic: NetDAM (618 ns avg for SIMD DRAM reads), RoCE BALBOA (~1.5–2.5 µs one-way), and optimized CXL-over-Ethernet FPGA prototypes (1.97 µs for remote load/store, further reduced to 415 ns with FPGA cache hits) (Fang et al., 2021, Wang et al., 2023, Heer et al., 27 Jul 2025). For distributed collectives (e.g., ring-based MPI-Allreduce over 4 nodes), NetDAM achieves 5.25× lower wall time than RoCEv2 and 1.9× higher throughput at <5% CPU utilization (vs. RoCEv2 ~30%) (Fang et al., 2021).

FPGA design resource utilization is moderate for core stacks (<18% LUTs for CXL Agent + cache + packet manager; <5% LUTs for RoCE BALBOA including on-datapath cryptography/ML-DPI modules) (Wang et al., 2023, Heer et al., 27 Jul 2025). For full 256-channel data acquisition, ListenToLight’s backend demonstrates <15% LUT/BRAM utilization, suggesting headroom for multi-port scaling (Villani et al., 26 Jan 2026). ASIC resource overhead of IRN is ≤3% (Mittal et al., 2018).

5. Programming Models, API, and Integration

100 GbE RDMA exposes one-sided (READ, WRITE) and two-sided (SEND, RECV) primitives, orchestrated through user- and kernel-space verbs and registered memory regions. Modern FPGA/SmartNIC platforms export C/C++-style user APIs (e.g., coyote::cThread, sgEntry in BALBOA; netdam_read/netdam_reduce_scatter in NetDAM), enabling explicit QP management, scatter-gather I/O, batched verbs, and offload hooks (Fang et al., 2021, Heer et al., 27 Jul 2025).

NetDAM’s instruction model encodes opcode, address, and data in the UDP payload; programmable ALUs process standard and user-defined operations in hardware, with pipeline-ordered request/completion queues for host synchrony (Fang et al., 2021).

Data acquisition architectures (e.g., ListenToLight) embed ERNIC IPs for direct AXI-BRAM integration and ring buffers, with batched RDMA WRITE posts via user-space DAQ daemons, minimizing OS involvement and achieving high deterministic throughput (Villani et al., 26 Jan 2026).

In disaggregated memory settings, CXL-over-Ethernet mappings bridge native x86 load/store usage to remote DRAM via CXL–AXI/Ethernet pipelines, hiding protocol translation and queuing, and retaining native semantics to the CPU (Wang et al., 2023). Application code requires little or no modification: real-world integration is demonstrated with standard workloads (MPI, in-memory analytics, GPU offload).

6. Applications, Use Cases, and System-Level Impacts

100 GbE RDMA is pivotal in distributed deep learning, memory pooling, large-scale data acquisition, memory disaggregation, and in-network computation.

Distributed collective operations benefit from line-rate offload and in-network compute (NetDAM).
High-throughput streaming (e.g., ultrafast optoacoustic/ultrasound imaging) exploits ring-buffered RDMA to achieve uninterrupted multi-gigabyte/sec data flows, scaling to hundreds of channels (Villani et al., 26 Jan 2026).
SmartNICs enable line-rate pre-processing for recommender systems, in-place analytics, crypto, and DPI—all without host CPU intervention (RoCE BALBOA) (Heer et al., 27 Jul 2025).
Disaggregated architectures use 100 GbE RDMA as the high-bandwidth link for rack-scale memory extension, achieving remote memory access latency of 1.97 µs (sub-500 ns with on-FPGA cache), with up to 72 Gbps host-visible throughput (Wang et al., 2023).

Scalability is facilitated via multi-QP batching, multi-port link aggregation, and programmable flow control, with typical resource use allowing substantial parallelism within current FPGA generations (Heer et al., 27 Jul 2025, Villani et al., 26 Jan 2026).

7. Limitations, Trade-offs, and Future Directions

Key trade-offs involve protocol complexity (e.g., header lengths in RoCE v2 vs. custom minimal stacks), ecosystem maturity (commercial driver/API support for ASICs vs. custom user logic for open FPGA solutions), and resource consumption (programmatic datapath costs vs. performance). Disabling PFC removes congestion collapse risk but mandates robust per-flow loss recovery and congestion control (IRN, DCQCN, BPFC) (Hoefler et al., 2023, Mittal et al., 2018).

Projected advances include:

Next-generation 100+ GbE RDMA standards with condensed headers, hybrid lossless/lossy transport classes, and fine-grained programmable congestion/telemetry (Hoefler et al., 2023).
Deeper SmartNIC and in-network pipeline programmability for analytics, storage, and transactional compute (Heer et al., 27 Jul 2025).
Transparent integration of remote load/store (CXL over Ethernet) with hardware coherency below 1 µs (Wang et al., 2023).
Multi-port, multi-ERNIC scaling, and dynamic batching for streaming DAQ and real-time feedback (Villani et al., 26 Jan 2026).

A plausible implication is that “buffer-free” streaming and programmable in-network compute will characterize future exascale, data-centric, and AI workloads, shifting the bottleneck from the host boundary to network programmability and end-to-end memory orchestration.

References:

(Fang et al., 2021, Mansour et al., 2018, Hoefler et al., 2023, Wang et al., 2023, Mittal et al., 2018, Villani et al., 26 Jan 2026, Heer et al., 27 Jul 2025)