ResBench: A Comprehensive Framework for Evaluating Database Resilience

Abstract: Existing database benchmarks primarily focus on performance under ideal running environments. However, in real-world scenarios, databases probably face numerous adverse events. Quantifying the ability to cope with these events from a comprehensive perspective remains an open problem. We provide the definition of database resilience to describe its performance when facing adversity and propose ResBench, a benchmark for evaluating database resilience. This framework achieves automation, standardization, and visualization of the testing process through clear hierarchical decoupling. ResBench simulates adverse events and injects them during normal transaction processing, utilizing a module to gather multiple metrics for the evaluation model. We assess database resilience across eight dimensions: throughput, latency, stability, resistance, recovery, disturbance period, adaptation capability and metric deviation. All the results are presented to users via a user-friendly graphical interface. We demonstrate the execution process and result interpretation of ResBench using two types of adversity datasets.