Interaction-Explicit Action Spaces

• Interaction-Explicit Action Spaces are defined by their explicit representation of interaction semantics, affordances, and outcome goals, enhancing agent interpretability and efficiency.
• They are constructed using techniques like affordance segmentation, latent embedding, and hierarchical decomposition to facilitate precise and sample-efficient control.
• Empirical studies show that these spaces reduce sample complexity in robotics and improve multi-modal decision-making in language, vision, and multi-agent applications.

Interaction-Explicit Action Spaces are formalizations and implementations of action sets in learning and control systems where actions are parameterized not merely by kinematic or syntactic outputs, but by their explicit interaction semantics with the environment, other agents, or objects. This paradigm moves beyond generic motion or token-level command spaces by embedding affordances, interaction primitives, or outcome-driven transformations directly into the agent’s action repertoire. The motivation stems from the observation that both embodied and abstract agents operate most efficiently when their action representations are aligned with core interaction structure, yielding interpretable, sample-efficient, and compositional control policies across domains including robotics, vision, natural language, and multi-agent systems.

1. Formal Definitions and Motivations

Interaction-explicit action spaces are distinguished from traditional, motion-centric or token-centric spaces by the explicit inclusion of interaction semantics, outcome goals, or affordance parameters within the action definition. In robotics, this includes joint control vectors augmented by force/torque or impedance parameters:

$a_t = [v_t, \omega_t, f_t]^T \in \mathbb{R}^{6+k}$

where $v_t$ is a velocity target, $\omega_t$ is an angular velocity, and $f_t$ is an explicit force or interaction command (Aljalbout et al., 2024). In embodied navigation and manipulation, high-level discrete actions such as $\mathrm{Open}$, $\mathrm{Close}$, or $\mathrm{Slice}$—conditioned on perceivable target objects—constitute the manipulation subset of the action space (Nagarajan et al., 2020). In language and multi-environment agents, interaction-explicitness is realized by the union of standard language tokens and environment-specific invocation or tool-routing actions (Yue et al., 8 Oct 2025).

The primary motivation is to bridge the gap between what an agent “can do” and the structure of its action representation. By aligning actions with affordances or explicit interaction modes, learning becomes more interpretable, exploration covers semantically meaningful regions, and downstream planning tasks are accelerated (Nagarajan et al., 2020, Aljalbout et al., 2024, Wang et al., 2023).

2. Methodologies for Constructing Interaction-Explicit Action Spaces

Multiple methodologies have emerged, each grounded in different formalism:

• Affordance-Based Segmentation: Agents learn per-pixel or region-based affordance maps indicating the probability of interaction success for each action type, feeding these maps to the policy for efficient exploration (Nagarajan et al., 2020).
• Embedding and Latent Mode Factorization: Unsupervised or weakly supervised learning constructs low-dimensional embeddings (e.g., $\omega \in \Omega \subset \mathbb{R}^n$) representing closed-loop interaction policies (body-affordances), clustering trajectories or action pairs into semantically distinct “interaction modes” (Guttenberg et al., 2017, Wang et al., 2023, Song et al., 2020).
• Hierarchical Action Decomposition: Hierarchical frameworks generate actions at multiple semantic levels (e.g., coarse action sketches and fine-grained controls), using feedback between levels and observation prediction to ensure alignment between intended interactions and resulting environment dynamics (Zhu et al., 21 Nov 2025).
• Template-Based and Tool-Routing Expansion: In language agents, the action space is expanded to include parser-level templates and explicit tool invocation actions, enabling seamless switching between natural language reasoning and external environment interaction (Yue et al., 8 Oct 2025, Ammanabrolu et al., 2020).
• Sequentialization/Binarization for Huge Action Spaces: For combinatorial or high-dimensional action sets, a micro-action decomposition (e.g., binarization) enables large or non-Markovian spaces to be managed as explicit sequences of sub-actions, clarifying interaction structure (Majeed et al., 2020).

Specific network architectures, such as U-Nets for affordance segmentation (Nagarajan et al., 2020), multi-modal transformer stacks (Faure et al., 2022), and actor-critic models for RL (Yue et al., 8 Oct 2025, Aljalbout et al., 2024), are chosen to support explicit interaction parameterization.

3. Empirical Evaluation and Benchmarking

Quantitative evidence from benchmark environments affirms the superiority and practical impact of interaction-explicit spaces:

• In 3D environment exploration, interaction-explicit agents discovered 1.33× more unique object-action pairs than object-centric coverage baselines, with a 42% reduction in sample complexity to reach 50% final task coverage (Nagarajan et al., 2020).
• Robotic manipulation tasks demonstrated 3× improvements in sample efficiency and 20–30% higher sim-to-real transfer rates when explicit force parameters are included in the action space (Aljalbout et al., 2024).
• For LLMs augmented with expanded routing and tool actions (ExpA), downstream task performance increased by up to 11.9% absolute accuracy in calculator-augmented benchmarks and achieved perfect accuracy on small sorting tasks by efficiently learning algorithmic decision trees with minimal interactions (Yue et al., 8 Oct 2025).
• In weakly supervised or self-supervised settings, latent embedding methodologies yielded semantically clustered interaction spaces with >0.9 reliability (probability new pairs sampled from clusters share true interaction semantics), supporting robust data augmentation and generalization (Song et al., 2020).
• Ablations consistently revealed that stripping interaction-explicit parametrization or feedback led to degraded coverage, exploration efficiency, or semantic alignment in downstream tasks (Nagarajan et al., 2020, Zhu et al., 21 Nov 2025).

4. Applications Across Domains

Interaction-explicit action spaces are utilized in diverse problem classes:

• Embodied Agents and Robotics: Efficient learning of manipulation skills, object affordances, or trajectory planning by integrating both kinematic and force interaction parameters leads to more robust, interpretable, and transferable policies (Aljalbout et al., 2024, Nagarajan et al., 2020, Zhu et al., 21 Nov 2025).
• Video-Based and Multimodal Action Detection: Structured attention over persons, objects, hands, and temporal context produces feature representations explicitly encoding interactions, resulting in improved action detection and classification (Faure et al., 2022).
• Language Agents and Tool-Using LLMs: By internalizing routing and environment-specific actions, LLMs natively reason over both linguistic and interaction primitives, yielding strong performance in hybrid reasoning and contingent planning (Yue et al., 8 Oct 2025, Ammanabrolu et al., 2020).
• Simulation and Data Augmentation: Interaction-explicit embeddings serve as the backbone for unsupervised or semi-supervised data augmentation pipelines, leading to high-diversity, high-fidelity action-response synthesis with minimal supervision (Song et al., 2020, Wang et al., 2023, Rybkin et al., 2018).
• General RL with Large Action Spaces: Sequentialization/binarization reduces the sample complexity and planning requirements in large or history-based RL problems, allowing principled aggregation and surrogate MDP construction (Majeed et al., 2020).

5. Theoretical Implications and Representational Properties

The adoption of interaction-explicit design confers several representational advantages:

• Interpretability: Each action or embedding is mapped to either a semantic affordance primitive (“open door”) or a target outcome in world or sensor space, enhancing policy understanding (Nagarajan et al., 2020, Guttenberg et al., 2017).
• Compositionality: Embedding frameworks (e.g., CLASP) show how minimality and composability enforced in latent codes yield state-independent, group-like action representations, facilitating temporally extended planning and interpolation between policies (Rybkin et al., 2018, Guttenberg et al., 2017).
• Efficiency and Good Exploration: By associating actions tightly with valid or novel interactions, exploration algorithms achieve denser reward signals, avoid wasting effort on physically impossible or semantically void actions, and structure search in functionally relevant subspaces (Nagarajan et al., 2020, Aljalbout et al., 2024).
• Transferability: Learned interaction modes or affordances generalize to novel objects or environments, as the latent or explicit parametrization encodes higher-level relationships rather than instance-specific dynamics (Wang et al., 2023, Guttenberg et al., 2017).

6. Limitations, Challenges, and Future Directions

Despite strong empirical and theoretical grounding, several challenges persist:

• Scalability and Coverage: The quality of learned interaction spaces depends critically on the coverage of training data and the expressiveness of the embedding; small datasets or environments with rare affordances can limit cluster purity or generalization (Song et al., 2020, Wang et al., 2023).
• Complexity and Initialization: In LLMs and high-DOF robots, integrating new interaction actions (tools, external modules) or embedding initialization strategies remains an open problem for lifelong and scalable learning (Yue et al., 8 Oct 2025, Guttenberg et al., 2017).
• Hybrid and Hierarchical Integration: There are relatively few demonstrations of end-to-end learned, hierarchical interaction-explicit action spaces combining both discrete high-level interaction modes and low-level continuous parameters; principled methods for such integration remain an active area (Zhu et al., 21 Nov 2025).
• Evaluation Metrics: Unlike action-label accuracy, comprehensive evaluation of interaction-explicit spaces must disentangle coverage, precision, diversity, and goal-alignment, necessitating domain-specific metrics (e.g., IAT-test/IAT-train, success rate, weighted entropy) (Song et al., 2020, Wang et al., 2023).
• Physical and Safety Constraints: Explicit force/impedance parametrization requires careful consideration of safety, actuation limits, and robust error handling for both simulation and sim-to-real transfer (Aljalbout et al., 2024).

7. Comparative Summary of Key Approaches

Approach / Domain	Explicit Parametrization	Key Reference(s)
Affordance Segmentation for RL	Object-parameterized high-level verbs	(Nagarajan et al., 2020)
Latent Embedding (Body-Affordance)	Low-dim $\omega$ covering sensor space	(Guttenberg et al., 2017)
Force/Impedance Augmentation	Per-step force/torque/impedance vectors	(Aljalbout et al., 2024)
Hierarchical Coarse-to-Fine Action	Goal, sketch, intermediate/fine action	(Zhu et al., 21 Nov 2025)
LLM ExpA/Routing Actions	Explicit routing/tool invocation	(Yue et al., 8 Oct 2025)
Paired-Embedding Unsupervised IAT	Interaction-adjacent latent clusters	(Song et al., 2020)
Action-Mode Latent Factorization	Mode $z$ conditionals + affordance	(Wang et al., 2023)

This synthesis underscores that Interaction-Explicit Action Spaces constitute a central principle unifying affordance-driven RL, interpretable multi-modal action detection, hierarchical planning, and tool-augmented reasoning. Their design and implementation are domain-specific but universally prioritize alignment between what the system can do and the manifold of semantically meaningful interactions available in its world.