Structured Representation

Abstract: Invariant representations are core to representation learning, yet a central challenge remains: uncovering invariants that are stable and transferable without suppressing task-relevant signals. This raises fundamental questions, requiring further inquiry, about the appropriate level of abstraction at which such invariants should be defined, and which aspects of a system they should characterize. Interpretation of the environment relies on abstract knowledge structures to make sense of the current state, which leads to interactions, essential drivers of learning and knowledge acquisition. We posit that interpretation operates at the level of higher-order relational knowledge; hence, invariant structures must be where knowledge resides, specifically, as partitions defined by the closure of relational paths within an abstract knowledge space. These partitions serve as the core invariant representations, forming the structural substrate where knowledge is stored and learning occurs. On the other hand, inter-partition connectors enable the deployment of these knowledge partitions encoding task-relevant transitions. Thus, invariant partitions provide the foundational primitives of structured representation. We formalize the computational foundations for structured representation of the invariant partitions based on closed semiring, a relational algebraic structure.

• The paper introduces a novel paradigm by positing that invariant representations emerge from structured partitions in abstract knowledge spaces rather than at the object level.
• It formalizes chaining, choice, and closure using closed semirings, linking algebraic operations to the construction and decomposition of relational graphs.
• Applications in MiniGrid demonstrate that partitioning knowledge spaces with Jordan blocks and antichains can enhance planning and guide effective state transitions.

This paper proposes a novel approach to representation learning, positing that invariant representations reside not at the object level, but within abstract knowledge spaces as structured partitions. These partitions are defined by the closure of relational paths, forming the bedrock where knowledge is stored and learning occurs. The core idea is to move beyond simply learning stable features from raw data to uncovering the inherent relational structures that govern an agent's understanding and interaction with the world.

The authors argue that interpretation of the environment, a crucial precursor to interaction, relies on such abstract knowledge structures. To achieve this, representations must satisfy three key requirements, dubbed the "three C's":

1. Chaining: Sequentially composing relations to form composite relations, primarily within partitions.
2. Choice: Selecting among multiple viable alternatives, often through querying mechanisms.
3. Closure: Ensuring consistency within partitions, defining their boundaries, and enabling redirection if paths are inconsistent.

These principles are formalized using closed semirings, an algebraic structure $(S, \oplus, \otimes, ^*, 0, 1)$.

• $\oplus$ (addition-like operation) typically represents choice or parallel composition (e.g., union of relations).
• $\otimes$ (multiplication-like operation) typically represents chaining or serial composition (e.g., relational composition).
• $0$ and $1$ are identity elements for $\oplus$ and $\otimes$ respectively.
• The crucial $^*$ operation (closure) intuitively represents the sum of all possible chainings of an element (e.g., $a^* = 1 \oplus (a \otimes a^*)$). In graph terms, this corresponds to reflexive-transitive closure, identifying all reachable states.

Implementing Structured Relational Computation

The paper details how these abstract concepts can be implemented:

1. Knowledge Representation as Graphs: Abstract knowledge is modeled as directed graphs where nodes are abstract characteristic features or entities, and edges are relations. For binary relations, an adjacency matrix $R$ (where $R[i,j] = 1$ if $i \ r \ j$, else $0$) can represent this.
   • Matrix addition ($R_1 \oplus R_2$) corresponds to the union of relations (choice).
   • Matrix multiplication ($R_1 \otimes R_2$) corresponds to relational chaining.
   • The transitive closure $R^* = \bigcup_{i=1}^n R^i$ (where $n$ is the number of nodes) identifies all reachable pairs of nodes. This can be computed using algorithms like Floyd-Warshall.
2. Identifying Invariant Partitions:
   • Directed Acyclic Graphs (DAGs) and Posets: The paper often assumes the underlying relational structure can be, or can be reduced to, a DAG. A DAG defines a partially ordered set (poset) $(\mathcal{P}, \preceq)$.
   • Zeta and Möbius Transforms: For posets, the zeta function $\zeta(x,y) = 1$ if $x \preceq y$ and $0$ otherwise, forms the zeta matrix $Z$. The zeta transform $F = Zf$ acts as a partial relational closure operator.

   • Jordan Normal Form: The paper proposes using the Jordan normal form of the relation matrix (or a related matrix) to identify these invariant partitions. A matrix $A$ is decomposed as $A = PJP^{-1}$, where $J$ is a block-diagonal matrix consisting of Jordan blocks.

     • Each Jordan block $J_n(\lambda)$ is of the form:

       $$J_n(\lambda) = \begin{bmatrix} \lambda & 1 & 0 & \cdots & 0 \ 0 & \lambda & 1 & \ddots & \vdots \ \vdots & \ddots & \ddots & \ddots & 0 \ \vdots & & \ddots & \lambda & 1 \ 0 & \cdots & \cdots & 0 & \lambda \end{bmatrix}$$

     • In the context of a DAG's relation matrix, each Jordan block corresponds to a chain of nodes (a path). These blocks $\mathcal{Z}_i$ (denoted $J$ in matrix notation) represent the fundamental invariant partitions of the knowledge space. Learning is then conceptualized as discovering these blocks.

* Antichains: An antichain is a subset of nodes in a poset where no two nodes are related by the partial order (i.e., mutually unreachable via directed paths within the graph). Maximal antichains are the largest such sets. These are identified from the transitive closure by finding nodes that are incomparable. Antichains often highlight points where transitions between partitions (connectors) are necessary.

The combination of Jordan block partitions (representing internal coherent flows) and antichains (highlighting boundaries and connection points) offers a robust way to segment the knowledge space.

1. Querying and Interaction:
   • Forward Query: "Where can I reach from state $X$?" (Corresponds to finding successors in the relational graph).
   • Backward Query: "From which prior states could I have arrived at state $Y$?" (Corresponds to finding predecessors). These queries operate on the established partitions, enabling reasoning within a block or identifying the need to transition to another.
2. Connectors for Inter-Partition Flow:

   While partitions $\mathcal{Z}_i$ represent self-contained units of knowledge with coherent internal relational flows, tasks often require moving between these partitions. Connectors serve as interfaces that mediate these transitions.

   • They define preconditions and postconditions for moving from one partition block to another.
   • In the Jordan decomposition $A = PJP^{-1}$, the columns of $P$ are generalized eigenvectors. These can be related to the connectors.
   • Connectors can be parameterized (e.g., by $\lambda_{ij}$ values) to represent the strength or feasibility of a transition between block $i$ and block $j$. This allows for "what-if" scenarios and controllability analysis.

Practical Application: MiniGrid Navigation

The paper illustrates these concepts using the MiniGrid obstructed maze environment, which includes rooms, objects (balls, boxes, keys), and doors (some locked).

1. Feasible Relation Matrix: A binary matrix is precomputed, encoding feasible relations like in_room, is_adj, can_be_moved, can_contain (box contains key), open_door (key opens matching door, or agent opens unlocked door).
2. Partition Discovery:
   • Jordan Blocks: Applied to this matrix, Jordan blocks identify chains of relations/actions. For example, a block might represent the sequence of actions within a single room or a path leading to acquiring an item. The source and sink nodes of each block define its entry and exit points.
   • Maximal Antichains: In this environment, maximal antichain nodes often correspond to critical decision points or "bottlenecks," such as locked doors. To pass a locked door, one must chain through other relations (e.g., get key from box, use key on door), effectively traversing different conceptual blocks via connectors.
   • Figure \ref{fig:blocks} in the paper shows that antichain nodes often interleave Jordan block boundaries, suggesting they pinpoint where connectors are needed.

Implementation Considerations and Advantages:

• Computational Complexity:
  • Jordan block decomposition is typically $\mathcal{O}(n^3)$ for an $n \times n$ matrix (representing $n$ entities/states). This is a precomputation step.
  • Once partitions $\mathcal{Z}_i$ are established, reasoning can be localized. If the knowledge base changes, only affected partitions might need recomputation.
  • For planning (e.g., in an MDP context with $n$ states, $m$ actions, $p$ partitions), standard forward planning is $\mathcal{O}(mn^2)$ per step. With structured partitions, if transitions mainly occur between $p$ partitions via $c$ connectors, complexity might be reduced to roughly $\mathcal{O}(cp^2)$ for inter-partition planning, plus intra-partition costs. This is beneficial if $p \ll n$ and $c$ is sparse.
  • Recomputing a single partition of size $n_i$ might be $\mathcal{O}(n_i^3)$.
• Deployment:

1. Define Entities and Relations: Identify the core entities and the types of relations between them relevant to the domain. 2. Construct Relation Matrix: Populate a matrix representing these relations. This might involve precomputation, learning from data, or expert knowledge. 3. Compute Partitions: Apply Jordan decomposition (or similar techniques like finding strongly connected components if cycles are allowed and meaningful, then DAG condensation) to identify invariant blocks. Identify antichains from the transitive closure of the (potentially condensed) graph.

# Pseudocode for conceptual flow import numpy as np from scipy.linalg import jordan_form # For numeric matrices # For symbolic/boolean matrices, custom algorithms are needed for path-based structures # R is the adjacency matrix of the relation graph # For simplicity, assume R leads to a DAG structure or use condensation # R_transitive_closure = compute_transitive_closure(R) # e.g., Floyd-Warshall # Identify Jordan Blocks (conceptual for relational structures) # For a boolean matrix representing a DAG, Jordan blocks correspond to path structures. # One might use algorithms to find longest paths, or decomposition into path covers. # partition_blocks = find_jordan_like_blocks(R) # Identify Antichains # antichains = find_maximal_antichains(R_transitive_closure) # Example: Representing a relation matrix # entities = ["keyA", "doorA", "boxA", "room1_entry"] # R = np.zeros((len(entities), len(entities))) # R[entities.index("keyA"), entities.index("doorA")] = 1 # keyA can_open doorA # R[entities.index("boxA"), entities.index("keyA")] = 1 # boxA can_contain keyA

4. Define Connectors: Specify rules or learn parameters for transitions between these blocks, potentially guided by antichain analysis. 5. Implement Query Mechanism: Develop functions for forward and backward queries that operate over these partitioned structures.

• Potential Limitations:
  • Scalability of matrix operations for very large numbers of entities.
  • Defining the "right" level of abstraction for entities and relations can be challenging and domain-dependent.
  • Handling dynamic changes in the relational structure itself (i.e., when the "knowledge" changes fundamentally) might require significant recomputation.

In summary, the paper provides a theoretical framework and initial computational methods for learning structured representations based on invariant relational partitions. By focusing on algebraic closure and the interplay of chaining, choice, and closure within a semiring formalism, it offers a path towards AI systems that can build more robust, transferable, and interpretable models of their environment through abstract knowledge structures. The use of Jordan blocks and antichains provides concrete ways to identify these knowledge partitions and the necessary interfaces (connectors) between them, as demonstrated in the MiniGrid example.