High Bandwidth Flash (HBF) Overview

Updated 12 January 2026

HBF is a NAND flash architecture that uses die stacking, TSVs, and DDR synchronous signaling to deliver terabyte-scale capacity with high-bandwidth performance.
It achieves up to 800 GB/s aggregate bandwidth and 1.6 TiB capacity through advanced techniques like multi-channel/way interleaving and HBM-style host interfaces.
Designed for read-oriented workloads in LLM inference and high-performance accelerators, HBF balances energy efficiency, cost, and thermal management.

High Bandwidth Flash (HBF) is a class of NAND flash memory interface and packaging architectures designed to deliver dramatically higher bandwidth per package—comparable to or exceeding high-bandwidth memory (HBM)—while maintaining the terabyte-scale capacity and cost structure of NAND flash. HBF addresses the critical bottlenecks of bandwidth and capacity in memory-bound applications, most notably in the inference phase of LLMs and high-performance, data-intensive accelerators. Key proposals in this domain leverage die-stacked NAND, double-data-rate synchronous signaling, and controller integration techniques to achieve these ends, with further enhancements via multi-channel/way interleaving, near-memory processing, and direct on-die acceleration.

1. Architectural Principles of High Bandwidth Flash

HBF’s fundamental organizational principles draw from the high-bandwidth die stacking and parallel interface designs of HBM and adapt them for the non-volatile, page-oriented nature of modern NAND flash.

Die Stacking and TSVs (Through-Silicon Vias): HBF packages comprise multiple 3D-NAND dies vertically stacked, each die connected to a controller base die via TSVs. The base die, fabricated in a logic process, embeds all per-channel controllers, error correction (ECC), wear-leveling engines, and PHY circuitry for high-speed parallel transfer (Ma et al., 8 Jan 2026).
HBM-Style Host Interface: The package exposes hundreds to thousands of pins, supporting multi-Gb/s signaling per pin. The PHY and pinout mirror those of HBM, enabling direct connection to existing HBM controllers on accelerators or the adoption of variants via CXL or PCIe (Ma et al., 8 Jan 2026).
DDR Synchronous Flash I/O: At the die and channel level, high-bandwidth signaling is achieved via double-data-rate (DDR) synchronous interfaces. All data transfers occur on both rising and falling edges of a Data Valid Strobe (DVS) signal, coordinated by an on-chip delay-locked loop (DLL). This architecture keeps unchanged the conventional flash pinout, ensuring backward and footprint compatibility (Chung et al., 2015).
Way and Channel Interleaving: Controllers support multiplexing over “ways” (parallel access to multiple flash chips per channel) and multi-channel striping (separate parallel buses), scaling aggregate bandwidth to saturate the host interface (Chung et al., 2015).

2. Quantitative Performance Metrics and Design Equations

HBF architectures are characterized by the following quantitative metrics:

Capacity: A typical HBF stack achieves ~1.6 TiB capacity, an order of magnitude larger than HBM4 (~100 GiB per stack) (Ma et al., 8 Jan 2026).
Bandwidth: Aggregate per-stack bandwidth is targeted at 400–800 GB/s, with <80 GB/s cited as a conservative lower bound. Single-channel DDR synchronous designs achieve 2–3× bandwidth gains over conventional single-data-rate (SDR) flash interfaces (Chung et al., 2015).
Power Efficiency: Per-stack power consumption is in the 20–80 W range, corresponding to bandwidth-per-watt >6.4 GB/s/W and capacity-per-watt >6 GiB/W (Ma et al., 8 Jan 2026).
Latency & Granularity: Typical random read latency is 10 µs (compared to DRAM’s 10–100 ns) with transfer granularity at the page level (tens of kB per access) (Ma et al., 8 Jan 2026).

Key equations:

Per-channel bandwidth $BW = \text{Bus\_width} \times (\text{edges\_per\_cycle}) / t_P$
Stack bandwidth $B_{stack} \simeq N_{dies} \times BW_{per\_channel}$
Total capacity $C_{stack} = N_{dies} \times C_{die}$

In practice, both read and write speeds show 1.65–2.76× and 1.09–2.45× improvements, respectively, over conventional designs in SLC-type NAND (similar for MLC). Way-interleaving further amplifies performance, with up to 2.75× speedup in read and 2.45× in write bandwidth in 16-way SLC configurations (Chung et al., 2015).

3. Comparative Analysis: HBF vs. HBM, DRAM, and Conventional Flash

HBF uniquely bridges the gap between DRAM, HBM, and conventional flash, as illustrated below:

Technology	Capacity/Stack	Bandwidth/Stack	Latency	Write Endurance	Suitability for LLM Inference
HBM4	10–100 GiB	~300–800 GB/s	~100 ns	Unlimited	BW optimal, insufficient cap.
DDR5	~64 GiB	~20–50 GB/s	~50 ns	Unlimited	Cap. OK, BW insufficient
Flash-Card	~4 GiB	~0.1 GB/s	~10 µs	$10^4$ – $10^5$ cycles	BW/cap. too low
HBF Stack	~1.6 TiB	~300–800 GB/s	~10 µs	$10^3$ – $10^4$ cycles	Optimal for static LLM weights

HBF achieves TB-scale capacity and HBM-class bandwidth at moderate cost, restricted to read-oriented or infrequently written data due to NAND endurance limits. DDR5 and conventional flash fall short on bandwidth, while HBM’s capacity is multiplication-limited by cost and package area (Ma et al., 8 Jan 2026).

4. System Integration and Software Considerations

Physical Integration: HBF stacks distribute around the accelerator on an interposer, matching HBM topologies. The host accelerator uses standard HBM PHYs or slight derivatives, enabling straightforward adoption (Ma et al., 8 Jan 2026).
Software and Data Placement: Due to flash’s high small-access latency and page granularity, software frameworks must:
1. Align accesses to large page chunks (tens of KB).
2. Prefetch weight tiles into high-speed SRAM or DRAM buffers.
3. Stream decode kernel inputs from these buffers to hide flash latency.
Memory Coherence: HBF is ideal for static, read-only data such as model weights or large corpora. Since writes are rare or infrequent, complicated coherence mechanisms (needed when multiple nodes update memory) are not required—dynamic data stays in DRAM/HBM (Ma et al., 8 Jan 2026).
Energy Efficiency: At high degrees of way interleaving, HBF reduces per-byte energy by approximately 20–30% compared to the conventional interface, reaching 0.48 nJ/byte at sixteen-way SLC interleaving (Chung et al., 2015).
Backward Compatibility: Pinout and physical footprint remain unchanged relative to prior flash designs: no new pins introduced, and backward-compatible fallback to single-data-rate (SDR) timing is supported via optional controller logic (Chung et al., 2015).

Innovations in high-bandwidth flash are complemented by research into near-memory processing and flash-integrated accelerators:

On-Die Compute and FlashAbacus: FlashAbacus integrates multiple flash channels and dies directly with an array of lightweight processors connected via a high-speed on-chip network. By localizing kernel execution near flash and offloading I/O stack logic (including page mapping and locking), FlashAbacus attains ~127% higher bandwidth and 78% lower energy consumption compared to host-system PCIe+NVMe approaches (Zhang et al., 2018).
Parallelism Modeling: Exploiting plane-level, die-level, and channel-level parallelism, aggregate bandwidth of such accelerators is $B_{total} = N_c \times N_{planes} \times b_{plane}$ , with scales up to $\sim6.2$ GB/s across 4 channels (with $t_{rd}\sim81~\mu$ s read latency per 8kB TLC page). Effective throughput is limited by the slowest of compute and I/O paths (Zhang et al., 2018).
Scheduling and Access Control: Kernel scheduling exploits both inter-kernel and fine-grained intra-kernel parallelism, orchestrated via dependency DAGs and range-locking data structures to maintain flash consistency and performance isolation (Zhang et al., 2018).

6. Trade-Offs, Design Challenges, and Research Directions

Endurance Constraints: The limited program/erase cycles of NAND ( $10^3$ – $10^4$ in HBF stacks) necessitate restricting HBF to infrequently updated data. Wear-leveling and bad-block management mitigate, but cannot eliminate, these endurance ceilings (Ma et al., 8 Jan 2026).
Controller and PHY Complexity: Implementing robust ECC, wear-leveling engines, and dozens of high-speed PHY lanes in the logic base die adds area and design complexity. Standardization (e.g., HBF-PHY) is required for ecosystem adoption (Ma et al., 8 Jan 2026).
Latency Boundaries: The page-based granularity and 10 µs access latency of NAND persist even with high interface bandwidth. Emerging architectural and software research targets improved small-block read performance, retiling of weight matrices, and/or compressive techniques to reduce latency (Ma et al., 8 Jan 2026).
Thermal Management: 3D flash stacking has lower dynamic power than DRAM but poses new thermal challenges that need co-design of heat-sinks and package airflow to assure reliability (Ma et al., 8 Jan 2026).

Research directions include:

Determining optimal DRAM:HBF ratios for inference nodes
Automated weight placement/orchestration for LLMs
Architecting flash with improved segment-based I/O for hybrid workloads
Exploring HBF scaling and efficiency in mobile or edge inference.

High Bandwidth Flash thus establishes a novel operating point for memory system architects: it enables direct, scalable, and power-efficient access to terabyte-scale, read-oriented datasets at bandwidths previously reserved for volatile memory systems, particularly empowering large model inference and high-throughput data analytics workflows (Chung et al., 2015, Ma et al., 8 Jan 2026, Zhang et al., 2018).