Looking at Vector Space and Language Models for IR using Density Matrices

Abstract: In this work, we conduct a joint analysis of both Vector Space and LLMs for IR using the mathematical framework of Quantum Theory. We shed light on how both models allocate the space of density matrices. A density matrix is shown to be a general representational tool capable of leveraging capabilities of both VSM and LM representations thus paving the way for a new generation of retrieval models. We analyze the possible implications suggested by our findings.