Extend Mnemis to Support More Data Modalities

Develop methods to extend the Mnemis memory framework to support additional data modalities beyond text across its base graph and hierarchical graph components, enabling both System-1 similarity search and System-2 global selection to operate over multimodal content.

Background

Mnemis organizes and retrieves long-term memory using a base graph (episodes, entities, edges) and a hierarchical graph (categories) to enable two complementary routes: System-1 similarity search and System-2 global selection. The current implementation and experiments focus on text-based conversational histories.

The authors explicitly state that, despite strong results, important directions remain open, including expanding the framework to support more data modalities. This would require adapting ingestion, representation, and retrieval to handle non-text inputs within both graph structures.