CauSight: Learning to Supersense for Visual Causal Discovery

Abstract: Causal thinking enables humans to understand not just what is seen, but why it happens. To replicate this capability in modern AI systems, we introduce the task of visual causal discovery. It requires models to infer cause-and-effect relations among visual entities across diverse scenarios instead of merely perceiving their presence. To this end, we first construct the Visual Causal Graph dataset (VCG-32K), a large-scale collection of over 32,000 images annotated with entity-level causal graphs, and further develop CauSight, a novel vision-LLM to perform visual causal discovery through causally aware reasoning. Our training recipe integrates three components: (1) training data curation from VCG-32K, (2) Tree-of-Causal-Thought (ToCT) for synthesizing reasoning trajectories, and (3) reinforcement learning with a designed causal reward to refine the reasoning policy. Experiments show that CauSight outperforms GPT-4.1 on visual causal discovery, achieving over a threefold performance boost (21% absolute gain). Our code, model, and dataset are fully open-sourced at project page: https://github.com/OpenCausaLab/CauSight.

• The paper introduces CauSight, a framework that infers cause-and-effect relationships among visual entities using the annotated VCG-32K dataset.
• It employs a tree-of-causal-thought strategy with Monte Carlo Tree Search and reinforcement learning to optimize reasoning trajectories.
• Results demonstrate a 21% recall gain over GPT-4.1 and enhanced cross-domain generalization on datasets like Objects365.

CauSight: Learning to Supersense for Visual Causal Discovery

Introduction

The paper "CauSight: Learning to Supersense for Visual Causal Discovery" (2512.01827) introduces a novel framework for visual causal discovery, a task where models are required to infer cause-and-effect relationships among visual entities in diverse scenarios. This task transcends traditional visual recognition, demanding models to understand not only what is present in a scene but also why events occur. To support this goal, the authors present the Visual Causal Graph dataset (VCG-32K), comprising over 32,000 images annotated with causal graphs at the entity level. The proposed model, CauSight, leverages VLMs to perform causally aware reasoning and demonstrate superior performance in visual causal discovery.

Figure 1: A comparison between VLMs that understand (a) scene graph, which specifies spatial relations between entities; (b) causal graph, which captures causal mechanisms between entities. Genuine reasoning requires discovering causal relations.

Methodology

The CauSight framework integrates three main components to facilitate visual causal discovery:

1. Training Data Curation: VCG-32K provides entity-level causal graph annotations derived from popular datasets like MS-COCO and Objects365, ensuring a robust grounding for causal inference.
2. Tree-of-Causal-Thought (ToCT): This component synthesizes reasoning trajectories using a tree-based strategy where region selection, entity recognition, and causality orientation are executed in a loop. The Monte Carlo Tree Search (MCTS) algorithm supports this process by exploring multiple reasoning paths, ultimately generating high-quality trajectories.
3. Reinforcement Learning: A specialized causal reward structure guides reinforcement learning, refining the model’s reasoning policy. The GRPO algorithm is employed for policy optimization without requiring a separate value function, enabling efficient adaptation to visual causal structures.

   Figure 2: The two-stage annotation pipeline of VCG-32K: bounding box refinement and causal relationship labeling.

Results

Experimental analysis showcases CauSight's significant advantages in visual causal discovery, particularly when compared to proprietary models such as GPT-4.1 and OpenAI o3. Notable improvements include a 21% absolute gain in recall over GPT-4.1, emphasizing CauSight's efficacy in constructing coherent causal graphs from visual data.

Figure 3: Illustration of a single synthesized reasoning trajectory. The teacher model can repeatedly execute three key actions to extend the reasoning trajectory.

Additionally, CauSight demonstrated strong cross-domain generalization on the Objects365 dataset subset, outperforming baseline models that lacked explicit causal reasoning frameworks. The ability to integrate causal prior knowledge through ToCT and the RL stage, while maintaining detection and reasoning balance, underpins its superior performance.

Implications and Future Directions

CauSight represents a pivotal step towards integrating causal reasoning in VLMs, emphasizing the importance of understanding causal relationships for both practical applications like robotics and autonomous systems, and theoretical developments in AI interpretability and intelligence. Future research could explore expanding model capabilities to incorporate dynamic scenes or temporal causality, further aligning AI systems with human-like reasoning.

Figure 4: Model generalizability across three OOD benchmarks.

Conclusion

The paper details a comprehensive approach to advancing visual causal discovery by leveraging a novel dataset, VCG-32K, and proposing a causally aware model, CauSight. The model not only enhances the understanding of causality in visual contexts but also points toward broader implications in AI reasoning and decision-making. By grounding AI systems in causal inference, researchers can expect improved performance and generalization, setting a foundational precedent for future explorations in causal AI.