Similarity-based Random Partition Distribution for Clustering Functional Data

Abstract: Random partition distribution is a crucial tool for model-based clustering. This study advances the field of random partition in the context of functional spatial data, focusing on the challenges posed by hourly population data across various regions and dates. We propose an extension of the generalized Dirichlet process, named the similarity-based generalized Dirichlet process (SGDP)-type distribution, to address the limitations of simple random partition distributions (e.g., those induced by the Dirichlet process), such as an overabundance of clusters. This model prevents excess cluster production and incorporates pairwise similarity information to ensure accurate and meaningful clustering. The theoretical properties of the SGDP-type distribution are studied. Then, SGDP-type random partition is applied to a real-world dataset of hourly population flow in $500\text{m}$ meshes in the central part of Tokyo. In this empirical context, our method excels at detecting meaningful patterns in the data while accounting for spatial nuances. The results underscore the adaptability and utility of the method, showcasing its prowess in revealing intricate spatiotemporal dynamics. The proposed random partition will significantly contribute to urban planning, transportation, and policy-making and will be a helpful tool for understanding population dynamics and their implications.