Lessons Learned from Efforts to Standardize Streaming In SQL

Abstract: Acknowledging the reality of streaming languages and platforms overlapping with SQL and database systems, in 2019 INCTIS Data Management established an Expert Group with the focused mission to initiate the process of standardizing streaming support in SQL. Over time, the roster included companies like Actian, Alibaba, Amazon Web Services, Confluent, dbt Labs, Google, Hazelcast, IBM, Materialize, Microsoft, Oracle, Snowflake, SQLstream and Timeplus. For the span of more than one year, representatives of each company have presented key features of their streaming product or, in some cases, multiple streaming products. These were live technical Q&A sessions accompanied by summary or position papers, which are unquestionably valuable. As expected, substantial time was spent in clarifying what common terms meant in each system and setting up a glossary. These sessions were followed by clarification notes and debates, and decisions that appeared mandatory to allow further progress. This first phase was followed by the next phase, which consisted of the group meetings, in which the expert group (EG) agreed on main exit criteria topics that a streaming solution in SQL must address, and position papers and follow-ups were written and discussed. This paper summarizes these group efforts, up to the summer of 2023.