The Ubiquitous Skiplist: A Survey of What Cannot be Skipped About the Skiplist and its Applications in Big Data Systems

Abstract: Skiplists have become prevalent in systems. The main advantages of skiplists are their simplicity and ease of implementation, and the ability to support operations in the same asymptotic complexities as their tree-based counterparts. In this survey, we explore skiplists and their many variants. We highlight many scenarios about how skiplists are useful, and how they fit well in these usage scenarios. We also compare skiplists with other data structures, especially tree-based structures. Extensions to skiplists include structural modifications, as well as algorithmic enhancements and operations. We categorize the existing extensions, and summarize the skiplist variants that belong to each category. We present how data systems incorporate skiplist variants into many different application scenarios to serve various purposes. These data systems cover a wide range of applications, from data indexing to block-chain, from network algorithms to deterministic skiplists, etc. It illustrates an impactful and diverse applications of skiplists in various domains of data systems.

• The paper provides a comprehensive survey of skiplists, detailing their probabilistic design and logarithmic time complexities for key operations.
• It compares skiplists with tree-based structures and discusses enhancements like lock-free and multi-versioned implementations to boost concurrency.
• The study examines real-world applications in systems such as LevelDB and RocksDB, addressing hardware adaptations for NUMA and GPU environments.

Overview of Skiplists and Their Applications in Big Data Systems

The paper "What Cannot be Skipped About the Skiplist: A Survey of Skiplists and Their Applications in Big Data Systems" by Vadrevu, Xing, and Aref provides a comprehensive survey of skiplists, discussing their structure, operations, and diverse applications. Skiplists are noted for their simplicity and efficiency, supporting operations with asymptotic complexities comparable to tree-based data structures. This survey explores various extensions and optimizations aimed at enhancing their applicability in big data systems.

Skiplist Fundamentals

The basic skiplist is a probabilistic data structure consisting of multiple levels, each containing a linked list of nodes. The structure is advantageous owing to its logarithmic time complexity for searches, inserts, and deletes. The authors explore detailed descriptions and cost analyses of these operations, emphasizing the balance between memory usage and speed determined by the promotion probability factor, $p$.

Comparative Analysis

The skiplist is compared to B/B+-trees, highlighting its probabilistic nature versus the deterministic guarantees of trees. Notably, deterministic skiplists offer bounded costs even in worst-case scenarios and are aligned with structures like 2-3 and 2-3-4 trees in terms of performance. This highlights the skiplist's adaptability and potential parity with widely-used database indices.

Concurrency and Multi-Versioning

A substantial portion of the paper addresses concurrency in skiplists, discussing lock-based and lock-free variations that ensure scalability and linear performance in multi-threaded environments. The survey covers advanced implementations, such as optimistic locking and unrolled skiplists, alongside multi-versioned skiplists that enhance performance in transaction-heavy environments.

Taxonomy and Systems Utilization

A significant contribution of this survey is the taxonomy of skiplist variants, detailing their evolution and usage across different systems, such as in-memory components of LSMT databases and NUMA-aware databases. Notably, systems like LevelDB and RocksDB employ skiplists for concurrent insertion and retrieval operations while maintaining sorted order, illustrating their practical application in modern data systems.

Hardware Implications

The work explores skiplist adaptations for emerging hardware such as persistent memory and GPUs. By addressing challenges like the asymmetric nature of flash memory and the complexity of NUMA environments, the paper underscores the versatile adaptability of skiplists in accommodating new hardware specifications and performance requirements.

Special Skiplist Variants

The paper covers multi-dimensional and interval skiplists, pointing out their applicability in handling complex data structures and range queries. The relevance of these variants within multi-dimensional indexing systems, such as those used in geospatial data management, showcases the skiplist's flexibility.

Implications and Future Directions

By providing a rich overview of skiplists' theoretical underpinnings and practical implementations, the survey suggests several potential research directions. Future developments may focus on optimizing skiplists for new data access patterns and hardware configurations, ensuring their continued relevance and efficiency in emerging big data paradigms.

In conclusion, this survey is a critical resource for researchers and practitioners, offering insights into the broad utility and future prospects of skiplists in computer science and data systems. The authors successfully consolidate skiplist knowledge, driving further exploration and application in complex, scalable, and concurrent data environments.