Parallel Continuous Chain-of-Thought (PCCoT)

• PCCoT is a continuous latent reasoning method that employs parallel Jacobi iteration to update dummy thought tokens simultaneously, eliminating sequential dependency.
• It refines an upfront block of latent tokens over a few iterations, achieving nearly 45% faster training and 55% quicker inference compared to standard continuous CoT.
• Empirical evaluations on GSM8K datasets with GPT-2 and Llama models show PCCoT delivers improved accuracy, stability, and efficiency over traditional methods.

Parallel Continuous Chain-of-Thought (PCCoT) is a continuous chain-of-thought reasoning method in which latent thought tokens are updated with Jacobi iteration rather than generated one by one. In the formulation introduced in "Parallel Continuous Chain-of-Thought with Jacobi Iteration" (Wu et al., 23 Jun 2025), the model allocates dummy latent thought tokens upfront and refines all of them synchronously over a small number of iterations. The stated objective is to preserve the compactness of continuous latent reasoning while removing the sequential bottleneck that makes standard continuous CoT expensive to train and decode. The reported result is that, with a suitable choice of latent-token count and iteration count, PCCoT can achieve comparable or better performance than sequential continuous CoT while saving nearly 50% of training and inference time, with better stability and robustness during training (Wu et al., 23 Jun 2025).

1. Origins and problem setting

PCCoT is defined against the contrast between standard chain-of-thought and continuous chain-of-thought. In standard CoT, the model emits explicit natural-language reasoning tokens before the answer. In continuous CoT, reasoning is compressed into continuous latent thought tokens rather than discrete text. The benefit is token efficiency, but the cost is that latent thought generation remains sequential: each latent token depends on all previous latent thoughts, so the model must execute multiple dependent forward passes (Wu et al., 23 Jun 2025).

In the continuous-CoT setup described in the PCCoT paper, the query is $x = (x_1, x_2, \dots, x_n)$, a special token $x_{n+1} = \texttt{<bot>}$ is appended, and a transformer $f$ without prediction head operates on token embeddings $E_{x_i}$. The first latent thought token is

$h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$

Subsequent latent thought tokens are then produced sequentially:

$$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$

After generating $c$ latent thought tokens, the model appends $x_{n+c+2} = \texttt{<eot>}$ and decodes answer tokens autoregressively. This sequential dependence prevents parallel generation of latent tokens and is inefficient on modern hardware, where fixed-length, highly parallel execution is preferred. In the reported GPT-2 Small experiments on GSM8K-Aug, sequential continuous CoT required 24.91 hours of training and 0.443 s/batch for inference, measured on the question and CoT tokens only (Wu et al., 23 Jun 2025).

2. Jacobi iteration over latent thought tokens

PCCoT removes the sequential dependence by placing all latent positions in the sequence from the start and updating them in parallel. Given input $x = (x_1, \dots, x_n)$, the method appends $x_{n+1} = \texttt{<bot>}$ and then $x_{n+1} = \texttt{<bot>}$0 dummy latent tokens $x_{n+1} = \texttt{<bot>}$1 for $x_{n+1} = \texttt{<bot>}$2. Before answer decoding, the sequence is

$x_{n+1} = \texttt{<bot>}$3

Let $x_{n+1} = \texttt{<bot>}$4 denote the final-layer hidden state of token $x_{n+1} = \texttt{<bot>}$5 at iteration $x_{n+1} = \texttt{<bot>}$6. The initial forward pass computes

$x_{n+1} = \texttt{<bot>}$7

PCCoT then performs $x_{n+1} = \texttt{<bot>}$8 extra Jacobi iterations. At iteration $x_{n+1} = \texttt{<bot>}$9, the input embeddings for the question and $f$0 positions remain fixed, while the input embeddings for the latent positions are replaced by the previous iteration’s hidden states:

$f$1

The defining property is that all latent positions are updated simultaneously in each call to $f$2. The reasoning phase is therefore one initial pass plus $f$3 extra passes, with every pass operating on the full latent block in parallel rather than extending a chain one token at a time (Wu et al., 23 Jun 2025).

After the final iteration, the model appends $f$4 and uses the hidden states from iteration $f$5 as context for standard autoregressive answer decoding. During inference, the reasoning cost is therefore $f$6 full-sequence passes regardless of $f$7, because the latent tokens are processed in parallel within each pass. The paper characterizes sequential continuous CoT as requiring roughly $f$8 passes and PCCoT as requiring $f$9 passes for sequence length $E_{x_i}$0. The practical consequence is the intended regime $E_{x_i}$1, where speedup is obtained without discarding latent reasoning entirely (Wu et al., 23 Jun 2025).

3. Formal relations, special cases, and equivalence claims

A central feature of PCCoT is that it parameterizes several other continuous-reasoning variants through the choice of latent-token count $E_{x_i}$2 and extra-iteration count $E_{x_i}$3 (Wu et al., 23 Jun 2025).

PCCoT setting	Correspondence
$E_{x_i}$4	Implicit CoT (iCoT)
$E_{x_i}$5	Nearly equivalent to Pause Tokens
$E_{x_i}$6 with fixed $E_{x_i}$7	Equivalent computation graph to continuous CoT

The paper’s Theorem 1 states that the computation graph of PCCoT with $E_{x_i}$8 latent thought tokens and $E_{x_i}$9 extra iterations is equivalent to that of continuous CoT with $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$0 latent thought tokens if $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$1 (Wu et al., 23 Jun 2025). The proof sketch given in the paper proceeds inductively: after the first iteration, the states of tokens up to $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$2 match those of continuous CoT; after the $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$3-th extra iteration, the $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$4-th latent thought token in PCCoT matches the corresponding token in sequential continuous CoT; hence once $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$5, all $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$6 latent thought tokens coincide.

This equivalence theorem is narrower than a blanket claim that PCCoT should always imitate sequential continuous CoT. The same source reports that the best empirical regime does not require $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$7, and that large $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$8 can become unstable. A common misconception is therefore to treat PCCoT primarily as an exact parallel reimplementation of sequential continuous CoT. In the paper’s own empirical analysis, the practically important regime is instead an interpolation between one-pass latent reasoning and strict sequential equivalence, with moderate $h_{n+1} = f([E_{x_1}; \dots; E_{x_{n+1}}]).$9 and small $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$0 often performing best (Wu et al., 23 Jun 2025).

4. Training objective and computational pipeline

PCCoT uses the same training objective as continuous CoT and differs only in the internal computation graph. The training framework adopted in the paper is CODI, a self-distillation scheme with a teacher task and a student task. The teacher task uses standard CoT with explicit reasoning and answer tokens, optimized by cross-entropy on all reasoning and answer tokens. The student task replaces text reasoning with latent thought tokens and is optimized by cross-entropy on answer tokens only. Distillation is applied on the last token of the answer prompt, such as the ":" in "The answer is:", by minimizing the $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$1 distance between teacher and student prediction distributions (Wu et al., 23 Jun 2025).

The stated overall objective is

$$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$2

The PCCoT paper notes one deliberate departure from the CODI baseline: CODI can optionally apply an MLP to post-process latent thought hidden states, but that MLP is intentionally not used in the PCCoT experiments so that comparisons remain fair (Wu et al., 23 Jun 2025).

For each training example, the teacher pass performs explicit CoT decoding, produces teacher logits over reasoning and answer tokens, and incurs teacher cross-entropy loss. The student pass constructs the input with $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$3 and $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$4 latent tokens, runs the initial forward pass and $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$5 Jacobi iterations, appends $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$6, decodes answer tokens, computes student answer-only cross-entropy, and adds the $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$7 distillation loss on the last answer-prompt token. Gradients are then backpropagated through the full iterative computation graph (Wu et al., 23 Jun 2025).

The reported experimental configuration uses GPT-2 Small and Llama3.2-1B-Instruct on GSM8K-Aug and GSM8K-Aug-NL, with 385k training examples, 500 development examples, and 1,319 test examples. Optimization uses LoRA with rank 128, $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$8, dropout 0.1, and adaptation on attention and FFN modules. GPT-2 uses learning rate $$h_{n+i+1} = f([E_{x_1}; \dots; E_{x_{n+1}}; h_{n+1}; \dots; h_{n+i}]), \quad i = 1, \dots, c.$$9, weight decay 0.01, and 40 epochs; Llama-1B uses learning rate $c$0, weight decay 0.1, and 10 epochs. Both use AdamW, a cosine learning-rate schedule, batch size 128, and gradient clipping 1.0 (Wu et al., 23 Jun 2025).

5. Empirical behavior: efficiency, accuracy, stability, and latent dynamics

The paper’s main reported PCCoT configuration uses $c$1 latent tokens and $c$2 extra iterations. The continuous-CoT baseline uses $c$3 sequential latent tokens, and the Pause Tokens baseline uses 24 pause tokens. In GPT-2 Small experiments, PCCoT reaches $c$4 on GSM8K-Aug and $c$5 on GSM8K-Aug-NL, compared with $c$6 and $c$7 for continuous CoT. In Llama3.2-1B-Instruct experiments, PCCoT reaches $c$8 and $c$9, compared with $x_{n+c+2} = \texttt{<eot>}$0 and $x_{n+c+2} = \texttt{<eot>}$1 for continuous CoT. The paper summarizes these results as consistent outperformance of continuous CoT and other continuous-style baselines, with generally smaller standard deviations across runs (Wu et al., 23 Jun 2025).

Method	Training Time (h)	Inference Time (s/batch)
CoT	4.39	1.353
Implicit CoT	11.23	0.040
Pause Tokens	11.88	0.041
Continuous CoT	24.91	0.443
PCCoT	13.72	0.199

These timings are reported for GPT-2 Small on GSM8K-Aug, with training on $x_{n+c+2} = \texttt{<eot>}$2H800 and inference on $x_{n+c+2} = \texttt{<eot>}$3A6000; inference uses batch size 100 and counts only the question and CoT tokens. Relative to continuous CoT, PCCoT reduces training time from 24.91 to 13.72 hours and inference time from 0.443 to 0.199 seconds per batch, which the paper characterizes as approximately 45% reduction in training time and approximately 55% reduction in inference time (Wu et al., 23 Jun 2025).

The sensitivity analysis over $x_{n+c+2} = \texttt{<eot>}$4 and $x_{n+c+2} = \texttt{<eot>}$5 shows a distinct operating region. For fixed $x_{n+c+2} = \texttt{<eot>}$6, increasing $x_{n+c+2} = \texttt{<eot>}$7 improves performance sharply up to around $x_{n+c+2} = \texttt{<eot>}$8, after which performance often degrades and becomes unstable; for large $x_{n+c+2} = \texttt{<eot>}$9, performance fluctuates substantially. For fixed $x = (x_1, \dots, x_n)$0, increasing $x = (x_1, \dots, x_n)$1 increases accuracy fairly steadily, with small variance. Even at $x = (x_1, \dots, x_n)$2, PCCoT significantly outperforms Pause Tokens for any $x = (x_1, \dots, x_n)$3. The paper describes this as a “sweet spot” at moderate latent-token count and small iteration count, exemplified by $x = (x_1, \dots, x_n)$4 (Wu et al., 23 Jun 2025).

The paper also studies latent-token dynamics. Its equivalence theorem concerns computation graphs, not fixed-point convergence, so the authors measure mean squared error between latent tokens across consecutive iterations. In a randomly initialized model, latent tokens converge smoothly as iterations increase. After training, however, latent tokens do not converge even with many iterations; the MSE remains nontrivial and fluctuates. The conclusion given in the paper is that PCCoT’s success is not explained by classical convergence of Jacobi iteration to a fixed point, but by a learned iterative process that remains dynamically active (Wu et al., 23 Jun 2025).

Pairwise MSE analysis across latent-token indices shows that higher-index latent tokens tend to be more similar to one another, while trained PCCoT tokens are more differentiated than those in a randomly initialized model. For some configurations, notably $x = (x_1, \dots, x_n)$5, the paper reports an interleaved pattern in which odd-index latent tokens are more similar to each other and less similar to even-index tokens, and vice versa. No explanation is established in the paper; the stated interpretation is that non-trivial interdependencies exist among latent tokens (Wu et al., 23 Jun 2025).

Initialization experiments further characterize robustness. With $x = (x_1, \dots, x_n)$6, using an untrainable random $x = (x_1, \dots, x_n)$7 embedding reduces GPT-2 performance on GSM8K-Aug from $x = (x_1, \dots, x_n)$8 to $x = (x_1, \dots, x_n)$9, which the paper interprets as evidence that the model is robust to the initial latent embedding while still benefiting from a learned embedding. The same source also reports that perturbing the learned $x_{n+1} = \texttt{<bot>}$0 embedding after training collapses accuracy to 0 (Wu et al., 23 Jun 2025).

A further comparison evaluates PCCoT-trained weights under explicit CoT decoding rather than latent reasoning. On GPT-2 and GSM8K-Aug, CoT baseline performance is 44.1, PCCoT is 49.48, and PCCoT with CoT decoding is 50.42. With gold reasoning steps, CoT reaches 88.21, whereas PCCoT with CoT decoding and gold reasoning steps reaches 62.24. The authors interpret the improved ordinary CoT decoding from PCCoT-trained weights as evidence that the student task acts as a regularizer. On GSM8K-Aug-NL, by contrast, latent PCCoT reasoning outperforms PCCoT plus CoT decoding; the paper offers a plausible explanation that discrete CoT suffers from train-test mismatch under teacher forcing, whereas latent reasoning avoids discrete step-by-step exposure bias (Wu et al., 23 Jun 2025).

6. Position within the broader literature

PCCoT belongs to a broader line of work on parallelization and continuous reasoning, but its specific mechanism is distinct from several nearby proposals. A cautionary comparison is provided by work on Parallel Context Windows (PCW), which parallelizes demonstrations by splitting them into windows with blocked cross-window attention and repeated positional embeddings. That study found a simple weighted-sum ensemble baseline comparable to PCW on most classification benchmarks and documented “Chain-of-Thought deterioration” on HotpotQA, with higher rates of question misinterpretation and false reasoning under parallel windows (Yang et al., 2023). This suggests that parallelization alone is not sufficient for coherent reasoning, and that PCCoT’s gains should be attributed specifically to synchronous latent-token updates rather than to a generic benefit of parallel CoT.

Other contemporaneous continuous-space methods parallelize reasoning in different ways. SoftCoT++ generates multiple latent thought trajectories for a single input by using multiple specialized initial tokens and contrastive learning to promote diversity among soft-thought representations; it is explicitly framed as scaling latent thoughts at the thinking stage and as generating multiple latent reasoning paths simultaneously (Xu et al., 16 May 2025). FastCoT, by contrast, operates in discrete token space and uses parallel decoding to create approximate future rationale tokens, reporting nearly 20% inference-time savings with negligible performance drop on its evaluated settings (Zhang et al., 2023). These methods address related efficiency or scaling questions, but they differ from PCCoT’s Jacobi-style refinement of a fixed block of latent tokens.

Theoretical work on continuous CoT provides a broader interpretive frame for PCCoT. One line, formulated as CoT2, establishes that continuously valued thought tokens can track multiple discrete traces in parallel, gives an embedding-dimension bound $x_{n+1} = \texttt{<bot>}$1 for robustly superposing $x_{n+1} = \texttt{<bot>}$2 states, and presents a one-layer transformer construction for a subset-sum-style task under continuous thought representations (Gozeten et al., 29 May 2025). A later training-dynamics analysis studies how superposition emerges in continuous CoT, arguing that a bounded index-matching logit balances exploration and exploitation and thereby enables the model to maintain multiple candidate reasoning traces in superposition (Zhu et al., 27 Sep 2025). These results do not define PCCoT itself, but they suggest a natural theoretical interpretation of why parallel latent reasoning can be effective.

Within that landscape, PCCoT can be understood as a practical algorithmic answer to a specific systems problem in continuous CoT: how to retain latent thought tokens while removing the sequential dependency that prevents parallel training and fast inference. Its distinguishing features are the use of dummy latent tokens, synchronous Jacobi updates, compatibility with the continuous-CoT training objective used in CODI, and an empirically validated small-$x_{n+1} = \texttt{<bot>}$3, moderate-$x_{n+1} = \texttt{<bot>}$4 regime in which performance, efficiency, and stability are jointly favorable (Wu et al., 23 Jun 2025).