Chain-of-Thoughts Schema

• Chain-of-thought schema is a reasoning paradigm where AI models generate sequences of modular, intermediate steps to mimic human thought processes.
• The approach employs non-autoregressive, DAG-based methods to generate and compare candidate reasoning steps, improving error correction and flexibility.
• Empirical results, as demonstrated by models like CANTOR, show state-of-the-art performance on math word problems with enhanced robustness and efficiency.

The chain-of-thought (CoT) schema is a reasoning paradigm in which LLMs or neuro-symbolic systems explicitly generate a sequence of intermediate steps—“thoughts”—before producing a final answer or decision. This structured approach separates solution derivation into modular, interpretable components, enabling more robust, flexible, and transparent problem-solving across domains such as mathematical reasoning, question answering, and multi-modal understanding.

1. Motivation and Conceptual Foundations

Standard autoregressive decoders construct reasoning chains by generating operations one-by-one in a fixed, sequential order. However, this imposes unnecessary constraints that may not align with the natural, unordered emergence of human thoughts during problem solving. The CoT schema seeks to relax these constraints by modeling reasoning as a process involving multiple candidate steps—generated either sequentially or simultaneously—which can then be compared, refined, and selectively chained to maximize logical consistency and solution robustness.

Formally, the CoT paradigm defines a solution to a reasoning problem as a sequence or set of modular operations:

$$Y = \{y_1, y_2, ..., y_{|Y|}\}, \quad \text{where each } y_i = \langle y_i^f, y_i^a, y_i^b \rangle$$

Here, $y_i^f$ is the operator (e.g., addition), $y_i^a$ and $y_i^b$ are operands chosen from constants, quantities in the input, or prior operations. Such a schema can be visualized as an execution structure (e.g., a chain, a DAG) encoding causal and logical dependencies (Shao et al., 2022).

2. Simultaneous Thoughts and Directed Acyclic Graphs

Moving beyond strict autoregression, frameworks like CANTOR encode reasoning as a directed acyclic graph (DAG) rather than a linear chain. Each vertex represents a potential reasoning step, emitting an operator and selecting operands from the pool of primitive quantities and previous operations:

$$Z = \{ z_1, ..., z_{|Z|} \}; \quad z_j = \langle p_j, z_j^f, z_j^a, z_j^b \rangle$$

Vertices are generated in parallel, and candidate reasoning steps are compared and chained post hoc to construct an equation or logical inference path. This non-autoregressive scheme allows the model to internally decide on dependency structure, supporting both linear and non-linear reasoning (e.g. branching, merging) (Shao et al., 2022). Advantages include:

• Absence of pre-defined decoding order
• Simultaneous exploration of diverse intermediary “thoughts”
• Enhanced error correction via comparison and selection among alternatives

3. Implementation Details: Decoding and Selection

In practical systems such as CANTOR, a shallow transformer-based decoder generates at each vertex:

• Operator logits: $$P_f(z_j^f | p_j, X) = \text{softmax}(W_f \cdot v_{p_j})$$
• Operand logits: e.g.,

$$P_a(z_j^a | p_j, X) = \text{softmax}\left( (W_q Q)^\top \cdot \left( W_a v_{p_j} \right) / \sqrt{d} \right)$$

All possible candidate operations are constructed simultaneously and then scored for consistency and logical validity. A dedicated root selection head chooses the most plausible “root” operation, from which the answer is computed by traversing the corresponding subgraph (the “chained thoughts”).

This approach supports both fully supervised (with explicit equations as supervision) and weakly supervised settings (supervising only on the final answer), as demonstrated in applications to MathQA, SVAMP, DROP, and DROP_num (Shao et al., 2022).

4. Empirical Performance and Robustness

Extensive empirical studies demonstrate:

• On complex math word problem datasets, CANTOR achieves state-of-the-art value accuracy (e.g., over 82–83% on MathQA) and significantly surpasses baseline models using sequential or structured decoders
• The model’s accuracy remains high even on problems involving more operations (i.e., deeper or more complex reasoning chains) and unseen equation templates
• On weakly-supervised tasks, integrating the candidate-DAG-based arithmetic module leads to substantial F1 improvements relative to tagging-based methods
• Ablation analyses show that modeling diverse candidate steps and ensembles, rather than enforcing a single linear chain, improves robustness and solution accuracy (Shao et al., 2022)

5. Theoretical and Practical Implications

The DAG-based CoT approach challenges the assumption that sequential autoregression best models complex human-like reasoning. By allowing for non-autoregressive, simultaneous generation and comparison of candidate reasoning steps, models can:

• Learn chains of logical dependencies without explicit order constraints
• More effectively mirror how humans generate and evaluate multiple possible solution paths before converging to an answer
• Outperform much larger LLM baselines (e.g., PaLM, GPT-3) using smarter architectural design rather than sheer scale

The strong performance of relatively small transformer-based models employing a DAG-based CoT schema underscores the critical role of reasoning architecture. In practical terms, this justifies investing in architectures that support non-linear, selective, and comparative reasoning—particularly in domains such as mathematical and symbolic computation. The paradigm is extensible to semi-supervised and weakly supervised settings and amenable to integration with external tools (e.g., calculators, verifiers).

6. Connections and Broader Outlook

The chain-of-thoughts schema aligns with a growing trend in AI reasoning towards structured, interpretable, and modular systems. The DAG-based approach outlined in CANTOR offers a blueprint for future CoT implementations: represent multiple candidate “thought paths” in parallel, then chain or select among them as needed. This model structure improves both the flexibility and robustness of AI systems, and supports greater transparency and error diagnosis—an important consideration for critical real-world deployments.

In sum, the schema articulated and realized by CANTOR establishes that the generation and selective chaining of simultaneous, modular reasoning steps via a DAG offers significant empirical and theoretical advances for numerical and logical reasoning systems (Shao et al., 2022).