vMODB: Unifying event and data management for distributed asynchronous applications

Abstract: Event-driven architecture (EDA) has emerged as a crucial architectural pattern for scalable cloud applications. However, its asynchronous and decoupled nature introduces challenges for meeting transactional requirements. Database systems, relegated to serving as storage engines for individual components, do not recognize transactions that span multiple components in EDAs. In contrast, messaging systems are unaware of the components' application states. Weaving such asynchronous and independent EDA components forces developers to relinquish transactional guarantees, resulting in data consistency issues. To address this challenge, we design vMODB, a distributed framework that enables the implementation of highly consistent and scalable cloud applications without compromising the envisioned benefits of EDA. We propose Virtual Micro Service (VMS), a novel programming model that provides familiar constructs to enable developers to specify the data model, constraints, and concurrency semantics of components, as well as transactions and data dependencies that span across components. vMODB leverages VMS semantics to enforce ACID properties by transparently unifying event logs and state management into a common event-driven execution framework. Our experiments using two benchmarks show that vMODB outperforms a widely adopted state-of-the-art competing framework that only offers eventual consistency by up to 3X. With its high performance, familiar programming constructs, and ACID properties, vMODB will significantly simplify the development of highly consistent and efficient EDAs.