Bucket-based Learned Index (BLI)

• Bucket-based Learned Index (BLI) is an in-memory index that employs a globally sorted, locally unsorted strategy to boost insertion throughput and enable lock-free concurrency.
• It partitions keys into buckets and segments while using adaptive model maintenance and split/merge operations to balance tree height, lookup/insert latency, and memory overhead.
• Empirical evaluations demonstrate that BLI significantly outperforms state-of-the-art indexes in mixed workloads, offering notable improvements in concurrent lookup and insertion operations.

The Bucket-based Learned Index (BLI) is an in-memory learned index structure designed to overcome the limitations of prior learned indexes that rely on strictly sorted in-node arrays. BLI employs a "globally sorted, locally unsorted" paradigm to optimize insertion throughput, reduce maintenance overhead, and enable lock-free concurrency, while balancing key metrics such as tree height, lookup/insert latency, and memory overhead. By organizing the index as a tree of segments and unsorted data buckets, and employing adaptive model maintenance and split/merge operations, BLI achieves high performance across a variety of workloads while supporting efficient concurrent access (Dong et al., 14 Feb 2025).

1. Motivation and Conceptual Foundations

Learned indexes such as RMI, ALEX, and LIPP utilize monotonic machine learning models to map keys to positions within strictly sorted arrays. While this design provides fast lookup via model prediction, it incurs severe drawbacks:

• Insertion overhead: Strict order maintenance often necessitates shifting large portions of arrays (ALEX) or chaining (LIPP), resulting in high insertion costs.
• Concurrency bottlenecks: In-node sorting and shifting preclude efficient lock-free concurrent updates, requiring global or fine-grained locks.
• In-node lookup inefficiency: Model prediction errors can cause long last-mile searches within nodes, degrading performance.

BLI addresses these limitations with a "globally sorted, locally unsorted" approach. The global key range is partitioned into non-overlapping Buckets with sorted pivots (e.g., minimal keys), ensuring efficient tree traversal. Each Bucket internally stores its (key, value) entries in an unsorted array, using a lightweight hint function (e.g., hash) for slot selection. This strategy decouples global order from local arrangement, facilitating fast inserts, efficient in-bucket probing, and frictionless lock-free concurrency (Dong et al., 14 Feb 2025).

2. Data Structure and Organization

The BLI structure is defined by the following parameters: total key count $N_\text{total}$, bucket capacity $B$, segment fanout $f$, and tree height $h \simeq \lceil \log_f (N_\text{total} / B) \rceil$. Key organizational concepts include:

• Bucket Capacity ($B$): Each leaf-level Data-Bucket (D-Bucket) is an unsorted array with $B$ slots, each holding (key, value, valid bit). The valid bit distinguishes live records.
• Fanout ($f$): Non-leaf segments contain $f$ child pointers, accompanied by a sorted pivot array.
• Initial Fill Ratio ($\alpha$): New Buckets are initialized at an occupancy ratio $\alpha \in (0,1]$, affecting memory overhead $O_\text{mem} = 1/\alpha$.
• Segments and Models: Each non-leaf Segment maintains a small linear regression model $m(k) = ak + b$ to predict the child index for a given key, alongside sorted child pointers.
• Global Layout: The index is a multi-level tree wherein each root or Segment recursively splits the key space, with D-Buckets forming the leaves. The split points (pivots) are globally sorted, but within each D-Bucket, keys are unsorted (Dong et al., 14 Feb 2025).

3. Core Operations and Adaptive Algorithms

Bulk Loading

BLI utilizes a bottom-up bulk loading protocol. An initial sorted input array is partitioned into $\lceil N_\text{total} / (\alpha B) \rceil$ D-Buckets filled to $\alpha B$, leaving $(1-\alpha)B$ slots empty per bucket. Each bucket's pivot and pointer are propagated upward as a new sorted array, recursively forming the hierarchical segment structure. Total time complexity is $O(N_\text{total})$.

Lookup

Lookup proceeds as follows:

1. Traverse segments from root to leaf, using model prediction $i_\text{pred}$, then NEAR_SEARCH to locate the child index whose pivot interval contains the query key.
2. At the D-Bucket, apply a constant-time hint function (e.g., key mod $B$) to suggest a slot; scan up to $B$ positions (modulo wraparound) to locate the key.

Time complexity for lookup (average case): $$O(h \cdot (t_\text{predict} + t_\text{pivot\_search}) + \mathbb{E}[\text{scan\_length}])$$, typically $O(h+1)$.

Insertion

Insertions find the appropriate D-Bucket and look for an empty slot using the hint function. If no empty slot exists, a split is triggered:

• Splitting: All current entries plus the new key are partitioned via median selection. New left/right buckets are created, pivots set, and a pointer swap updates the parent using Read-Copy-Update (RCU). Amortized insertion cost remains $O(1)$ per operation.

If a segment's accuracy degrades or it overflows, a Split/Merge Operation (SMO) is invoked:

• Segment Scaling: Increase child pointer array size if model is still within error tolerance.
• Split: Partition segment using a new model if error exceeds $\epsilon_\text{split}$.
• Merge: Collapse groups of adjacent segments if a lower complexity model achieves accuracy within $\epsilon_\text{merge}$.
• Model Retraining: Refit models where error drifts post-insertion.

Triggering conditions distinguish between passive (forced by overflows) and active (model error threshold) SMOs (Dong et al., 14 Feb 2025).

4. Concurrency Model and Implementation

BLI is designed for lock-free Single-Producer/Multi-Consumer (SPMC) environments.

• In-Bucket Lock-Free Updates: Each D-Bucket slot has a valid bit. The writer inserts key/value, issues a memory barrier, and sets the valid bit; readers check the valid bit before reading data, ensuring safety without locking.
• RCU-based Structure Modification: Bucket splits and segment-level SMOs allocate and prepare replacement nodes off the main path. Replacement is achieved via atomic pointer swaps, and memory reclamation is deferred via a grace period until all ongoing readers have completed.
• Concurrency Protocol: Readers and writers never block each other. The protocol ensures that concurrent lookups and inserts are safe under the SPMC model—there is one dedicated writer, with any number of concurrent readers.

This architectural design enables efficient scaling to high thread counts with negligible coordination overhead, as measured empirically (see Section 6) (Dong et al., 14 Feb 2025).

5. Analytical Cost Models and Tuning Trade-offs

BLI exposes several tunable parameters:

• Tree Height: $$h \simeq \lceil \log_f(N_\text{total} / (\alpha B)) \rceil$$
• Lookup Latency: $$L \simeq h \cdot (t_\text{predict} + t_\text{pivot\_search}) + \mathbb{E}[\text{scan\_length}]$$, with $$\mathbb{E}[\text{scan\_length}]\simeq 1/(1-\text{occupancy})$$
• Insert Latency: Amortized $O(L+1)$, with split cost $O(B)$ dispersed over $\Theta(B)$ insertions
• Memory Overhead: $O_\text{mem} = 1/\alpha$

Parameter choices impact performance:

Parameter	Increase Effect	Decrease Effect
$B$	$\downarrow h$, $\downarrow$ model error; $\uparrow$ in-bucket scan & range-query cost	-
$f$	$\downarrow h$; $\uparrow$ model/segment cost & pointer mem	-
$\alpha$	$\downarrow O_\text{mem}$, $\uparrow$ conflicts & splits	$\uparrow O_\text{mem}$, fewer splits

Practical tuning recommendations:

• For read-heavy workloads: larger $B$, moderate $f$, and $\alpha$ near 0.6–0.8.
• For write-heavy workloads: smaller $B$, smaller $\alpha$, and $f$ chosen to control $h$ and limit SMOs.

A plausible implication is that BLI can be rapidly reparameterized to meet workload-specific latency and space requirements (Dong et al., 14 Feb 2025).

6. Experimental Evaluation and Results

BLI was evaluated using 200 million key/value pairs (8B each) across datasets of varying key distributions: books (easy), fb (medium), and osm (hard). Hardware was an Intel Xeon E5-2670 v3 (2.3 GHz) with 128 GB RAM.

• Baselines: BLI was compared to ALEX, LIPP, PGM-Index, and FINEdex, including their lock-based concurrent variants.
• Workloads: Bulk-load of 100 million keys followed by 100 million operations under variable read:write ratios and up to 24 threads.

Key findings:

Metric	fb	osm	books
Mixed (50:50, single-threaded)	up to 2.21× vs. SOTA	up to 2.21× vs. SOTA	~1.3× vs. SOTA
Insert-only	up to 1.47× vs. LIPP/ALEX	up to 1.40× vs. LIPP/ALEX	-
Read-only	up to 1.37× vs. best	up to 1.15× vs. best	ALEX slightly faster due to smaller $h$
Multithreaded (24T, 7:3 R:W)	up to 3.91× vs. lock-based ALEX+/LIPP+	up to 3.91×	-
Bulk-load	5.6× vs. ALEX; 2.5× vs. LIPP	-	-
Memory overhead ($\alpha=0.6$)	$1.67\times$ data size	segments add $<1\%$	-

Operational breakdown:

• Get(): 13% segment traversal; 87% in-bucket scan.
• Put(): 75% insertion/hint/SMO; 25% memory allocation.

These experiments substantiate BLI’s theoretical advantages in concurrency and maintenance efficiency (Dong et al., 14 Feb 2025).

7. Strengths, Limitations, and Future Directions

Strengths

• High insertion throughput by eliminating array shifts/chains in leaf nodes.
• Lock-free SPMC concurrency enabled by unsorted in-bucket layout and RCU protocol.
• Fast, single-pass bottom-up bulk loading process.
• Explicit parameter tuning for trade-offs among index height, in-node costs, and memory footprint.

Limitations

• Range queries require additional per-bucket sorts, resulting in slower in-memory execution for such workloads.
• Optimal parameter tuning ($B$, $f$, $\alpha$) is workload-dependent and may require empirical profiling.
• Worst-case in-bucket scan is $O(B)$.

Future Directions

• Incorporation of order-preserving hint functions for S-Buckets to reduce segment lookup costs.
• Adaptive or heterogeneous bucket resizing across partitions to optimize for workload skew.
• Extension to out-of-core settings (e.g., SSDs, NVRAM) through I/O-aware batching.
• Deployment of more expressive models (splines) inside segments for data with extreme skew (Dong et al., 14 Feb 2025).