# Abstract

Large Reasoning Models (LRMs) have demonstrated remarkable capabilities by scaling up the length of Chain-of-Thought (CoT). However, excessively long reasoning traces pose substantial challenges for training cost and inference latency. While various CoT compression approaches have emerged to address this challenge, they face inherent trade-offs: token-level methods often disrupt syntactic and logical coherence, while step-level methods based on perplexity fail to reliably capture the logically critical reasoning steps because of the dilution of logical information. In this paper, we propose **ASAP** (**A**nchor-guided, **S**urpris**A**l-based **P**runing), a novel coarse-to-fine framework for CoT compression. ASAP first performs anchor-guided pruning to preserve the core reasoning structure, which efficiently reduces the search space for subsequent processing. Leveraging the insight that logical branching choices are concentrated at the onset of reasoning steps, it then enables logic-aware pruning by selecting logically essential reasoning steps based on a novel first-token surprisal metric. Finally, ASAP distills the models to autonomously generate and leverage these concise CoTs at inference time, enabling efficient reasoning. Experiments show that ASAP achieves state-of-the-art accuracy across multiple benchmarks while substantially reducing training and inference costs.

[IMAGE: Illustration of CoT pruning by ASAP. The Original CoT generated by LRMs exhibits two types of redundancy: (1) Structural Redundancy, such as digressive branches (highlighted in red dashed boxes), which are removed by Stage 1 Anchor-guided pruning; and (2) Logical Redundancy within valid paths. ASAP addresses the latter in Stage 2 by computing the surprisal of the first tokens of reasoning steps (marked in blue) to identify and retain only the critical cognitive pivots. (figures/motivation.pdf)]

# Introduction

The emergence of Large Reasoning Models, including OpenAI's o1 [jaech2024openai] and DeepSeek-R1 [guo2025deepseek], marks a paradigm shift in artificial intelligence. By scaling up Chain-of-Thought (CoT) reasoning [wei2022chain], these models demonstrate emergent capabilities in complex domains such as mathematics [sun2025survey], programming [shi2024code; yang2025elaboration; hu2025flowmaltrans], and logical reasoning [liu2025logical; zhang2025gam; zhang2025mmcot]. However, this performance comes at a prohibitive cost: reasoning traces often span thousands of tokens, introducing substantial latency and memory overhead. Crucially, these lengthy traces often contain substantial redundancy, such as over-explaining simple problems or superficially exploring multiple paths for complex ones [bi2024program; xie2025logic; wu2025more; qu2025survey]. For instance, the _Original CoT_ in Figure 1 contains tangential branches (highlighted in red dashed boxes), such as exploring an alternative manual implementation that is subsequently rejected ("_But that's more code..._"). Furthermore, the reasoning is punctuated by syntactic fillers that contribute little to the core logic. This observation raises a fundamental question: _Can we identify and retain only the "cognitive pivots" of reasoning while discarding the redundancy?_

A growing body of research has emerged on CoT compression for efficient reasoning [qu2025survey]. Token-level methods like TokenSkip [xia2025tokenskip] adapt general-purpose context compressors such as LLMLingua-2 [pan2024llmlingua] to prune non-informative tokens. However, indiscriminate token removal risks disrupting the syntactic integrity of the reasoning chain. To address this, step-level pruning methods like SPIRIT [cui2025stepwise] trim entire reasoning steps, thereby preserving structural coherence. Nevertheless, these approaches face a fundamental challenge: accurately estimating the logical importance of each step. They typically rely on fixed metrics like perplexity (PPL), which measures the overall predictability of a sentence. This holistic measure often dilutes the signal of critical logical leaps with the noise of syntactically predictable but logically trivial content.

In this work, we ground CoT compression from an information-theoretic perspective. Through an empirical analysis of 10 million reasoning tokens (detailed in Section 2), we find that the logical progression within a CoT sequence is not uniformly distributed; instead, its information density is highly concentrated at the beginning of each reasoning step---specifically within the first few tokens (blue-highlighted in Figure 1). These tokens, such as "_But_" (self-correction) or "_So_" (deduction, not continuation), serve as high-entropy **cognitive pivots**. By leveraging the surprisal of these initial tokens, we can distinguish between critical logical transitions and predictable elaborations.

Guided by this insight, we propose **ASAP** (**A**nchor-guided, **S**urpris**A**l-based **P**runing), a coarse-to-fine framework designed to preserve these high-information steps. ASAP in a two-stage cascade that directly addresses the two types of redundancies identified in our study (illustrated in Figure 1): First, it employs **Anchor-guided Pruning** to remove structural redundancies. By generating a concise step-by-step reasoning trace as a logical backbone, it identifies and prunes the irrelevant branches (e.g., the red boxes in Figure 1). Second, it performs **Surprisal-based Refining** to eliminate logic-sparse steps. Leveraging our First-Token Surprisal metric, this stage iteratively filters out steps acting as mere fillers while retaining the high-surprisal cognitive pivots. Finally, we distill these compact, logic-dense CoTs into a target model, enabling it to generate efficient reasoning chains.

We validate our approach through extensive experiments on the DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B [guo2025deepseek] across diverse domains. The results demonstrate that ASAP establishes a superior Pareto frontier between performance and efficiency. Notably, on the challenging LiveCodeBench v4_v5 benchmark, ASAP achieves **36.19%** Pass@1 while reducing token generation by **23.5%** and inference latency by **43.5%** compared to the strongest baseline.

Our main contributions are summarized as follows:

- We present an empirical analysis of the information concentration of CoTs, uncovering that the surprisal of the starting token for each CoT step is a more robust indicator of logical importance than perplexity.

- We propose **ASAP**, a novel CoT compression framework that combines structural alignment with information-guided refinement.

- Extensive experiments on multiple benchmarks demonstrate that models fine-tuned on CoTs pruned by ASAP achieve state-of-the-art accuracy while substantially reducing computational costs.

[IMAGE: Empirical analysis of 10M tokens from DeepSeek-R1-Distill-Qwen-32B. (a) The entropy distribution reveals a clear information concentration: first tokens (blue) exhibit significantly higher uncertainty (entropy) compared to body tokens (orange), which are highly deterministic. (b) The most frequent first tokens are a mixture of logical operators (e.g., Wait) and ubiquitous syntactic connectors (e.g., So). (c) High-entropy states filter out predictable fillers like So or Then, while exclusively highlighting cognitive pivots such as Perhaps, What, and Alternative. (figures/entropy_distribution.pdf, figures/plot_frequency_bar.pdf, figures/plot_entropy_bar.pdf)]

# Empirical Analysis

To investigate the intrinsic distribution of logical information in CoTs, we conducted a large-scale analysis on 10 million tokens generated by DeepSeek-R1-Distill-Qwen-32B [guo2025deepseek] across diverse reasoning benchmarks (AIME and LiveCodeBench [jain2024livecodebench]).

#### Information Concentration in CoTs.

We analyze the entropy distribution of the _first token_ of each reasoning step compared to all subsequent tokens. Entropy, in this context, quantifies the model's uncertainty regarding the next state transition [shannon1948mathematical; malinin2020uncertainty; kuhn2023semantic; wang2025beyond; cheng2025reasoning]. As illustrated in Figure 5(a), a distinct concentration is observed. Starting tokens (blue) exhibit a dispersed distribution with a significantly higher 90th percentile. In contrast, other tokens (orange) are heavily concentrated near zero entropy, indicating that once a reasoning step is initiated, its subsequent elaboration is largely deterministic and syntactically driven. This empirical evidence confirms that the logical branching points, where the model actively deliberates on the reasoning path, are structurally concentrated at the beginning of each step.

#### Identifying Cognitive Pivots with Entropy.

Having identified the informative start tokens, we perform a more in-depth analysis of real logical pivots among the start tokens. We aim to distinguish between superficial syntactic connectors and real logical pivots. Figure 5(b) presents high-frequency start tokens that appear in CoTs, which mix connectors ("_So_", "_Let_") with reasoning markers ("_Wait_", "_But_"). However, when focusing on tokens generated in high-entropy states (Figure 5(c)), a qualitative shift emerges. Predictable connectors like "_So_" and "_Then_" are effectively suppressed due to their low uncertainty. Instead, the distribution is dominated by terms representing cognitive pivots and state transitions, such as: 1) Exploration: "_Alternative_", "_Another_" (proposing hypotheses or new paths). 2) Causality: "_Because_", "_Since_" (providing formal justification). 3) Self-Correction: "_Perhaps_", "_What_" (indicating error detection or logic reversal).

This analysis demonstrates that high entropy is a robust indicator of logical salience. Since the actual next token is known in the given training sequence, we operationalize this insight by using _First-Token Surprisal_ as a proxy to identify and preserve these critical reasoning hops in our proposed framework [fu2025deep].

[IMAGE: The overall framework of ASAP. The pipeline consists of three phases: (1) In Stage 1, the LLM generates a "Direct Thought" from the (Question, Answer) pair. This acts as an anchor to prune the "Original CoT" into a "Coarse-grained Pruned CoT". (2) In Stage 2, we compute the First-Token Surprisal for each step in the coarse-grained pruned CoT. High-surprisal steps are retained, while low-surprisal fillers are pruned, yielding the final "Fine-grained Pruned CoT". (3) In Training Stage, the data with ASAP pruned CoTs is used to fine-tune the LRM for efficient inference. (figures/overview.pdf)]

# Methodology

## Overall Framework

Formally, we consider a supervised reasoning task defined by a dataset `latex $\mathcal{D} = \{(Q_i, C_i, A_i)\}_{i=1}^N$ `, where `latex $Q_i$ ` is the query, `latex $A_i$ ` is the predicted answer, and `latex $C_i$ ` represents the original CoT generated by the LRMs. `latex $C_i$ ` is a sequence of reasoning steps `latex $C_i = \{s_1, s_2, \dots, s_L\}$ `. Our goal is to compress each `latex $C_i$ ` into a concise pruned one `latex $C_i'$ ` such that `latex $|C_i'|\ll|C_i|$ ` while the LRMs maintains the quality of generated reasoning steps and answers when fine-tuned on the dataset `latex $\mathcal{D'} = \{(Q_i, C_i', A_i)\}_{i=1}^N$ `.

We propose **ASAP**, a coarse-to-fine framework tailored to the redundancy of "Original CoT" (`latex $C$ `), as illustrated in Figure 6. Stage 1 (Anchor-guided Pruning) reduces _structural redundancy_ (e.g., dead ends) by aligning the CoT with a generated logical backbone. The LLM generates a "Direct Thought" (`latex $\mathcal{P}$ `) from the `latex $(Q, A)$ ` pairs. `latex $\mathcal{P}$ ` acts as an anchor to prune the `latex $C$ ` into a "Coarse-grained Pruned CoT" (`latex $C_{coarse}$ `). Stage 2 (Surprisal-based Refining) reduces _logical redundancy_ (e.g., syntactic fillers) by filtering non-informative steps. We approximate the information of each step in `latex $C_{coarse}$ ` with their surprisal of start tokens and prune low-surprisal steps, yielding the final "Fine-grained Pruned CoT" (`latex $C'$ `). Finally, all `latex $\{(Q, C', A)\}$ ` are utilized to fine-tune the target model.

## Anchor-guided Pruning

Directly pruning raw CoTs is challenging due to the noise and unstructured digressions inherent in LLM reasoning [zhou2024can]. To address this, we first construct a high-level logical skeleton to narrow the pruning space.

#### Generate Direct Thoughts.

We prompt the LLM to infer a concise reasoning path called "Direct Thought" (`latex $\mathcal{P}$ `) based on the `latex $(Q, A)$ ` pair (see Appendix 8 for prompts). Unlike exploratory CoTs, `latex $\mathcal{P}$ ` is generated as a structured, step-by-step explanation that outlines how to derive the answer from the question, exemplified in Appendix 8. This `latex $\mathcal{P}$ ` acts as a reference anchor, outlining the least reasoning trajectory required to solve the problem.

#### Pruning with Pattern Matching.

Guided by the anchor `latex $\mathcal{P}$ `, we prompt the LLM to prune the original CoT `latex $C$ `. Specifically, the LLM is instructed to: 1) remove unnecessary reasoning steps from `latex $C$ `; 2) retain all key supporting content that aligns with the logic of `latex $\mathcal{P}$ `; and 3) crucially, preserve the original wording without introducing new information. The prompt used for pruning is shown in Appendix 8. The goal is to extract the subsequence of `latex $C$ ` that semantically aligns with `latex $\mathcal{P}$ ` while discarding irrelevant branches (as shown in the "Original CoT" block of Figure 6).

Crucially, to mitigate LLM hallucination during compression, we enforce an extractive constraint, which validates structural and semantic alignment with `latex $C$ `. Specifically, we design a pattern-matching algorithm that verifies whether each step in `latex $C_{coarse}$ ` corresponds to a matching step in `latex $C$ ` while preserving their original order. The matching is performed using Gestalt Pattern Matching [black2004ratcliff] as a text similarity metric. A pruning is considered valid only if all steps in `latex $C_{coarse}$ ` achieve a similarity score above a predefined threshold `latex $\tau$ ` when matched against sequential steps in `latex $C$ `. The full pattern-matching algorithm is detailed in Algorithm 1 (see Appendix 7). We leverage high-temperature sampling, which provides the necessary diversity to efficiently re-prompt failed cases, ensuring that a valid `latex $C_{\text{coarse}}$ ` can be eventually generated.

## Surprisal-based Refining

Following the coarse-grained pruning, the resulting `latex $C_{coarse}$ ` may still contain verbose steps that contribute little to the logic. Grounded in our empirical finding that logical information is concentrated (Section 2), we perform a meticulous, logic-aware refinement in `latex $C_{coarse}$ ` to identify more subtle redundancies within the core reasoning path.

#### First-Token Surprisal as Logical Importance.

We introduce _First-Token Surprisal_ as a novel metric to precisely quantify the logical importance of each step, enabling us to filter out the least informative ones and produce the final highly condensed CoT. Let a reasoning step `latex $s$ ` be a sequence of tokens `latex $s = (x_1, x_2, \dots, x_T)$ `. The informational value of `latex $s$ ` within the context of previous steps `latex $\mathcal{C}_{pre}$ ` is typically estimated by its joint probability. However, our analysis reveals that the _first token_ `latex $x_1$ ` serves as the "cognitive pivot" carrying the majority of the uncertainty. Therefore, we define the _First-Token Surprisal_ `latex $\mathcal{S}(s)$ ` as:

```latex
$$\mathcal{S}(s \mid \mathcal{C}_{pre}) = - \log P_{\theta}(x_1 \mid \mathcal{C}_{pre})$$
```

where `latex $P_{\theta}$ ` denotes the probability distribution of the LRM. A high `latex $\mathcal{S}(s)$ ` indicates a high-information transition (e.g., initiating a new deduction or self-correction), whereas a low score suggests a deterministic continuation or syntactic filler.

#### Pruning using First-Token Surprisal.

We formulate the fine-grained pruning as a constrained maximization problem. Our goal is to select a subset of steps `latex $S' \subset C_{coarse}$ ` that maximizes the total logical information subject to a length budget `latex $L_{max}$ `:

```latex
$$S^* = \operatorname*{arg\,max}_{S' \subseteq C_{coarse}} \sum_{s \in S'} \mathcal{S}(s) \quad \text{s.t.} \quad \sum_{s \in S'} \text{len}(s) \le L_{max}$$
```

This formulation explicitly prioritizes steps with high information density. To solve this efficiently, we employ a greedy iterative strategy. We calculate the surprisal score for all steps in `latex $C_{coarse}$ ` and iteratively remove the step with the lowest `latex $\mathcal{S}(s)$ `, while the relative order of steps in `latex $S'$ ` is preserved. The detailed procedure is provided in Algorithm 2 (see Appendix 7). This process yields the final fine-grained CoT `latex $C'$ `, which retains the critical "aha moments" [guo2025deepseek] while meeting efficiency constraints.

## Supervised Fine-tuning

Following the pruning, we construct the final training dataset `latex $\mathcal{D}' = \{(Q_i, \mathbf{C}_i', A_i)\}_{i=1}^N$ `. For each instance, we concatenate the pruned CoT (`latex $\mathbf{C}_i'$ `) and the final answer (`latex $A_i$ `) to form the complete target response `latex $R_i$ `. We then fine-tune the LRM to minimize the standard negative log-likelihood of the target response tokens, conditioned on the input question. Formally, the loss is defined as:

```latex
$$\mathcal{L} = - \sum_{i=1}^{N} \sum_{j=1}^{|R_i|} \log P_{\theta}(r_{i,j} | Q_i, r_{i,<j})$$
```

where `latex $r_{i,j}$ ` is the `latex $j$ `-th token of the target response `latex $R_i$ `, and `latex $\theta$ ` represents the parameters of the model being fine-tuned. This supervised fine-tuning process effectively distills the knowledge from our pruning framework into the model. By training on these compact, logically salient examples, the model learns to internalize efficient reasoning patterns.

# Experiments

## Experimental Setup

#### Models and Datasets.

All experiments are conducted on the DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B [guo2025deepseek], with DeepSeek-R1-Distill-Qwen-7B as the default backbone across all settings. For the code reasoning domains, we use the Python subset of the CodeForces-CoTs [openr1] dataset. For the math reasoning domain, we adopt the OpenR1-Math [openr1] dataset and randomly sample 10K instances to match the size of the code subset, ensuring a balanced comparison across domains. The datasets consist of high-quality Chain-of-Thought (CoT) samples generated by DeepSeek-R1, making it particularly suitable for training competitive reasoning tasks.

#### Benchmarks.

We evaluate our method on a suite of widely used benchmarks that cover both code generation and mathematical reasoning tasks. For code generation, we adopt HumanEval+ [chen2021evaluating; liu2023your], LiveCodeBench v1_v3, LiveCodeBench v4_v5 [jain2024livecodebench], and LeetCodeDataset [xia2025leetcodedataset]. For mathematical reasoning, we evaluate on GSM8K [cobbe2021training], MATH500 [hendrycksmath2021], AIME24, and AIME25.

#### Baselines.

We compare our method against a comprehensive set of baselines. **Zero-shot** refers to the original model without any task-specific fine-tuning. **Original** denotes the model fine-tuned on the uncompressed CoTs from the training data. Among compression approaches, Selective Context [li2023compressing] prunes redundant lexical units based on self-information; **LLMLingua-2** [pan2024llmlingua] distills GPT-4's token importance signals into a lightweight Transformer encoder trained as a token classifier; **TokenSkip** [xia2025tokenskip] learns to skip less informative tokens to achieve controllable compression; and **SPIRIT** [cui2025stepwise] identifies critical reasoning steps by measuring perplexity shifts. Except for the zero-shot setting, all methods involve fine-tuning on CoTs processed according to their respective compression strategies.

#### Metrics.

We evaluate both accuracy and inference efficiency of each approach across three metrics: **Pass@1 (Acc)**, which measures the percentage of problems correctly solved on the first attempt; **Tokens (Tok)**, which denotes the average number of tokens generated by the LRMs; and **Latency (Lat)**, which measures the average time (in seconds) required for the model generation.

## Main Results

The results show that the model fine-tuned on CoTs pruned by ASAP consistently achieves the best trade-off between accuracy and efficiency. It achieves the best accuracy while generating the fewest tokens, leading to the lowest generation latency.

We notice a clear distinction between token-level and step-level pruning strategies. Token-level baselines such as Selective Context, LLMLingua-2, and TokenSkip exhibit a significant performance degradation compared to the original CoTs. This is because the token removal disrupts the syntactic structure and semantic coherence of the original reasoning steps. Consequently, the fine-tuning data becomes fragmented and grammatically unnatural, making it difficult for the model to learn the intended logical flow of the CoT. Step-level methods, such as SPIRIT, perform significantly better than token-level pruning methods, due to the preservation of sentence-level integrity. While SPIRIT improves efficiency over the Original with comparable accuracy, our method achieves higher efficiency and accuracy at the same time. This improvement is particularly pronounced on the challenging LiveCodeBench v4_v5 benchmark: ASAP reduces the average number of generated tokens by **23.5%** (from 7892 to 6035) and lowers generation latency by **43.5%** (from 4.62s to 2.61s), while also achieving a **7.8%** improvement in accuracy (Pass@1 increases from 33.58% to 36.19%).

## Ablation and Analysis

#### Effect of Different Components.

To validate the contribution of each component, we conduct an ablation study on three model variants. 1) _w/o Anchor-guided Pruning_: which skips Stage 1 and applies only surprisal-based pruning to the original CoT. 2) _w/o Surprisal-based Refining_: which omits the surprisal-based refinement stage; and 3) _w/o Both Pruning_: equivalent to the original baseline, where the model is fine-tuned on the full, uncompressed CoT. The results show that both pruning stages are essential and mutually complementary for optimal accuracy and efficiency. First, removing the anchor-guided pruning leads to a drop in both accuracy and efficiency. While the accuracy decrease is modest, the generation latency increases by a substantial **76.2%** (from 2.61s to 4.60s), underscoring the importance of stage 1. Second, removing the surprisal-based refining results in a significant degradation across all metrics. The accuracy drops by **12.4%** (Pass@1 decreases from 36.19% to 31.72%) relative to the ASAP, and efficiency improvements are largely lost. This highlights that our surprisal-based pruning mechanism is essential to select the most critical steps.

| Variants                     | Acc       | Tok      | Lat      |
| ---------------------------- | --------- | -------- | -------- |
| **ASAP**                     | **36.19** | **6035** | **2.61** |
| w/o Anchor-guided Pruning    | 35.07     | 7735     | 4.60     |
| w/o Surprisal-based Refining | 31.72     | 8061     | 4.83     |
| w/o Both Pruning             | 30.97     | 8289     | 4.83     |

_Table: Ablation study of different pruning strategies on LiveCodeBench v4_v5. We report accuracy (Acc), average number of generated tokens (Tok), and average generation latency (Lat) measured in seconds._

#### Generalization to Different Architectures.

To validate the generalizability of ASAP, we replicate our main experiments on the DeepSeek-R1-Distill-Llama-8B. Following the same experimental protocol, we compare ASAP against three strong baselines: Zero-shot, Original, and SPIRIT. We observe consistent trends across all benchmarks. The results in the Llama3.1 series are highly consistent with our findings in the Qwen2.5 series, confirming the generalization of the ASAP. ASAP achieves the highest accuracy on both benchmarks, and the efficiency improvements are even more pronounced. On LiveCodeBench, for instance, ASAP not only surpasses the accuracy of the Original baseline (32.84% vs. 31.34%) but also generates **49.1%** fewer tokens and reduces latency by over **3x** (from 8.60s to 2.69s). This suggests that the ASAP is particularly effective in identifying and distilling the core reasoning patterns, validating its robustness and broad applicability for improving reasoning efficiency across different model families.

| Methods   | LCB Acc   | LCB Tok  | LCB Lat  | AIME Acc  | AIME Tok | AIME Lat |
| --------- | --------- | -------- | -------- | --------- | -------- | -------- |
| Zero-shot | 25.00     | 8508     | 8.90     | 33.33     | 8445     | 10.42    |
| Original  | 31.34     | 8202     | 8.60     | **36.67** | 8550     | 10.04    |
| SPIRIT    | 30.22     | 7913     | 8.45     | **36.67** | 8788     | 10.04    |
| ASAP      | **32.84** | **4175** | **2.69** | **36.67** | **5314** | **6.97** |

_Table: Experimental results of different methods with DeepSeek-R1-Distill-Llama-8B. We report accuracy (Acc), average number of generated tokens (Tok), and average generation latency (Lat) measured in seconds._

[IMAGE: Performance of ASAP on LiveCodeBench v4_v5 under different token budgets. (figures/budget_v4_v5.pdf)]

#### Impact of Token Budget.

To evaluate the scalability and resource sensitivity of our method, we analyze its behavior under varying inference-time token budgets (i.e., the maximum number of tokens to the model). We compare ASAP against the three strong baselines---SPIRIT, Original, and Zero-shot---across all benchmarks, and observe consistent trends. ASAP consistently outperforms all baselines across all budget settings. In particular, ASAP exhibits smooth performance scaling with respect to the token budget. We note that ASAP achieves superior performance-efficiency trade-offs. For example, ASAP with just an 8K token budget achieves higher accuracy than SPIRIT and Original at a much larger 12K budget. These results further validate the practical utility of ASAP in real-world scenarios.

#### Training Efficiency.

To quantify the training efficiency gains, we present results of the CodeForces-CoTs dataset. The results highlight the training efficiency advantage of the ASAP. By generating the most compact yet logically rich CoTs, our approach significantly reduces training overhead. Compared to the uncompressed baseline (Original), our method reduces the number of training tokens by **75.6%** and shortens training time by **60.7%**. These savings substantially exceed those achieved by all other baselines. ASAP enables a more resource-efficient training process, making it a practical and cost-effective solution for real-world deployment.

| Methods           | Tokens            | Time               |
| ----------------- | ----------------- | ------------------ |
| Original          | 13023             | 80.11              |
| Selective Context | 6722 (-48.4%)     | 63.41 (-20.9%)     |
| LLMLingua-2       | 6919 (-46.9%)     | 65.25 (-18.6%)     |
| TokenSkip         | 9813 (-24.6%)     | 77.27 (-3.6%)      |
| SPIRIT            | 6082 (-53.3%)     | 57.45 (-28.3%)     |
| ASAP              | **3178 (-75.6%)** | **31.48 (-60.7%)** |

_Table: Training efficiency comparison on CodeForces-CoTs dataset. We report the average number of tokens per sample and training time measured in seconds per step._

# Related Work

#### Chain-of-Thought and Advanced Reasoning.

Chain-of-Thought (CoT) prompting [wei2022chain] has evolved from heuristic prompting strategies [yao2023tree; lei2023boosting; ling2023deductive] to the training of specialized Large Reasoning Models (LRMs) like OpenAI's o1 [jaech2024openai] and DeepSeek-R1 [guo2025deepseek]. These models leverage reinforcement learning to scale test-time compute, generating lengthy reasoning traces to solve complex tasks [team2025kimi; yang2025qwen3; yu2025dapo; wang2025beyond; zhang2025cf; zhang2025tri]. Unlike prior works that enhance performance by scaling up CoT length, we focus on pruning redundancy to improve efficiency without compromising reasoning performance.

#### Context Compression for LLMs.

To mitigate the computational cost of long contexts, various compression techniques have been proposed [zhang2025hybridtoken]. Approaches like Selective Context [li2023compressing], LLMLingua series [jiang2023llmlingua; pan2024llmlingua], and LongCodeZip [shi2025longcodezip] employ information-theoretic metrics or small external models to filter redundant tokens. However, these methods typically treat input as unstructured text. Applying them directly to CoT often disrupts the syntactic and logical coherence required for valid reasoning, a limitation that our pruning aims to overcome.

#### Efficient Reasoning via Fine-Tuning.

Recent research has explored various efficiency mechanisms [qu2025survey], ranging from compressing thoughts into continuous latent representations [hao2024training; cheng2024compressed; shen2025codi] to compressing CoTs [kang2025c3ot; xia2025tokenskip; cui2025stepwise]. Approaches like TokenSkip [xia2025tokenskip] and SPIRIT [cui2025stepwise] reduce length by filtering tokens or steps based on heuristics or perplexity shifts. However, these metrics often struggle to differentiate between syntactic fluency and logical necessity. ASAP differs by combining anchor-guided structural pruning with first-token surprisal, offering a more robust proxy for cognitive pivots.

# Conclusion

In this paper, we address the inefficiency of Large Reasoning Models stemming from the structural and logical redundancies in Chain-of-Thought reasoning. Grounded in an information-theoretic perspective, our large-scale empirical analysis reveals a fundamental property of reasoning traces: Information Concentration, where the logical uncertainty is highly concentrated at the onset of reasoning steps. Guided by this insight, we propose ASAP. This coarse-to-fine framework first aligns the reasoning structure with a logical anchor and then refines it using a novel First-Token Surprisal metric. Extensive experiments across multiple benchmarks demonstrate that ASAP outperforms existing baselines, establishing a new state-of-the-art Pareto frontier between accuracy and efficiency. Our work highlights the potential of using information-theoretic signals for efficient reasoning. Future work will explore applying ASAP to online inference acceleration.

# Limitations

While ASAP demonstrates significant improvements in reasoning efficiency, we acknowledge several limitations. First, our method relies on the availability of a capable LLM to generate high-quality "Direct Thoughts" in Stage 1. If the anchor contains logical errors or hallucinations, it may misguide the subsequent pruning, although our pattern-matching constraint mitigates this risk. Second, our experiments primarily focus on code generation and mathematical reasoning. While we believe the principle of information concentration applies broadly, the effectiveness of ASAP on creative writing or commonsense reasoning tasks remains to be verified.

# Algorithms

**Algorithm 1: Pattern Matching Validation**

Input: Original CoT `latex $C$ `, Coarse-grained Pruned CoT `latex $C_{coarse}$ `, Threshold `latex $\tau$ `
Output: True if `latex $C_{coarse}$ ` is valid, False otherwise.

1. `latex $S_{origin} \leftarrow \text{SplitStepsByBlankLine}(C)$ `
2. `latex $S_{coarse} \leftarrow \text{SplitStepsByBlankLine}(C_{coarse})$ `
3. `latex $origin\_idx \leftarrow 0$ `
4. For each `latex $s_{coarse}$ ` in `latex $S_{coarse}$ `:
   - `latex $found\_match \leftarrow \text{False}$ `
   - While `latex $origin\_idx < |S_{origin}|$ `:
     - `latex $s_{origin} \leftarrow S_{origin}[origin\_idx]$ `
     - `latex $score \leftarrow \text{GestaltSimilarity}(s_{origin}, s_{coarse})$ `
     - If `latex $score \geq \tau$ `:
       - `latex $found\_match \leftarrow \text{True}$ `
       - `latex $origin\_idx \leftarrow origin\_idx + 1$ `
       - break
     - `latex $origin\_idx \leftarrow origin\_idx + 1$ `
   - If not `latex $found\_match$ `: return False
5. Return True

**Algorithm 2: Iterative Surprisal-based Pruning**

Input: Coarse-grained Pruned CoT `latex $C_{coarse}$ `, Max Tokens `latex $L_{max}$ `, Model `latex $M$ `, Tokenizer `latex $T$ `
Output: Fine-grained Pruned CoT `latex $C'$ `

1. `latex $S \leftarrow \text{SplitStepsByBlankLine}(C_{coarse})$ `
2. `latex $SurprisalScores \leftarrow \text{CalculateAll}(S, M, T)$ `
3. `latex $StepsToPrune \leftarrow \text{SortByScore}(S, \text{SurprisalScores})$ ` (ascending)
4. `latex $S_{current} \leftarrow S$ `
5. For each `latex $s_{prune}$ ` in `latex $StepsToPrune$ `:
   - `latex $S_{temp} \leftarrow S_{current} \setminus \{s_{prune}\}$ `
   - `latex $C_{temp} \leftarrow \text{Join}(S_{temp})$ `
   - If `latex $|T(C_{temp})| \leq L_{max}$ `:
     - `latex $S_{current} \leftarrow S_{temp}$ `
     - break
   - `latex $S_{current} \leftarrow S_{temp}$ `
6. `latex $C' \leftarrow \text{Join}(S_{current})$ `
7. Return `latex $C'$ `

# Prompt Templates and Generated Examples

We prompt the LLM to produce a direct thought: a concise, structured, step-by-step explanation that outlines how to derive the answer from the given question. A direct thought always consists of a small number of clear logical steps that directly contribute to the solution, and terminates with a single, explicit final answer.

**Prompt for generating direct thoughts:**

> Given a question, please tell me how to get this answer step by step.
> Question:
> {question}
>
> Answer:
>
> ```python
> {answer}
> ```
>
> Only return a detailed step-by-step solution (containing only "Step-by-Step Solution" and "Final Answer"). The detailed step-by-step solution is:

**Example direct thought** (for converting an integer to binary):

> **Step-by-Step Solution**
>
> 1. Understand the Problem: The task is to convert a given positive integer `n` into its binary representation without leading zeros. The constraints are lenient (n <= 10^6), so even a straightforward approach is efficient.
> 2. Input Handling: Read the integer `n` from standard input. Since the input consists of a single integer, we can directly use `input()` and convert it to an integer.
> 3. Binary Conversion: Python's built-in `bin()` function converts an integer to its binary string representation (prefixed with `0b`). To remove the prefix, slice the string from the third character onward (`[2:]`).
> 4. Output the Result: Print the resulting binary string. This ensures no leading zeros are present.
> 5. Efficiency: The approach is optimal because conversion and slicing both run in O(log n) time, and memory usage is minimal.
>
> **Final Code:**
>
> ```python
> n = int(input())
> print(bin(n)[2:])
> ```

**Prompt for coarse-grained pruning:**

> Compress the given thinking by referring to the provided solution. The goal is to remove irrelevant reasoning paths while retaining all content along the core reasoning path. Compression must be based on thinking, ensuring that the original wording and structure are preserved as much as possible. Follow these strict rules:
>
> 1. Use thinking as the foundation: Do not rewrite or replace its content with solution---only use solution to determine which parts are relevant.
> 2. Remove unnecessary reasoning: Aggressively remove alternative paths that are not part of the core reasoning path.
> 3. Retain key supporting content: Keep examples, reflections, and tests that help illustrate, verify, or analyze the core reasoning path.
> 4. Preserve original words: Do not paraphrase, reorder, or change any words.
> 5. Do not add new words: Do not introduce new concepts, symbols, or abbreviations.
>
> If you understand, compress the following thinking based on the given solution.
> Solution:
> {solution}
>
> Thinking:
> {think}
>
> The compressed thinking is:

# Implementation Details

#### Software and Hardware.

For fine-tuning, we utilized the unsloth library for its memory-efficient optimizations. For inference, we employed the vLLM engine to maximize throughput and efficiency. All experiments were conducted on NVIDIA H20 GPUs and Intel Xeon Platinum 8480+ CPUs.

#### Fine-tuning Configuration.

We performed full-parameter fine-tuning for all models in our experiments. Key hyperparameters included precision set to bf16, num_train_epochs set to 10, and a max_seq_length of 16384. We used a per_device_train_batch_size of 1 with gradient_accumulation_steps set to 16, resulting in an effective batch size of 16. For the optimizer, we used AdamW with a cosine_with_min_lr learning rate scheduler. The warmup_ratio was set to 0.03, and the scheduler's min_lr_rate was 0.1 of the peak learning rate. To stabilize training, we applied gradient clipping with a max_grad_norm of 0.2. Based on preliminary experiments, we set the peak learning rate to 4e-5 for the DeepSeek-R1-Distill-Qwen-7B and 2e-5 for the DeepSeek-R1-Distill-Llama-8B. Due to the high computational cost of full-parameter fine-tuning, the model is fine-tuned by a single run with a fixed random seed 42.

#### Inference and Evaluation Protocol.

All inference benchmarks were run using the vLLM engine with dtype set to bfloat16 and gpu_memory_utilization set to 0.9. To ensure deterministic and reproducible outputs, we set the sampling temperature to 0.0 and set enable_prefix_caching to False. The default token budget for generation is adjusted based on the task difficulty. Specifically, it is 2K for GSM8K, 4K for MATH500, 6K for HumanEval+, and 10K for AIME24, AIME25, LiveCodeBench, and LeetCodeDataset.

#### Baseline Details.

Following established practices, we used a consistent scoring model; as our primary model is DeepSeek-R1-Distill checkpoints, we employed DeepSeek-R1-Distill-Qwen-7B for all model-scoring tasks. To ensure a fair comparison, we standardize the input format across all methods by preserving the original question and final answer, and applying compression only to the CoT reasoning steps. To balance compression ratio and content retention, we set the target compression ratio to 0.5 for all baseline methods, except for TokenSkip, where we follow its original design that allows a controllable compression ratio between 0.5 and 1.0.

#### Hyperparameters for Our Method.

Our method involves several stages. For the LLM-guided Coarse-grained Pruning stage, we employed DeepSeek-V3 for economic reasons. When generating the direct thought `latex $\mathcal{P}$ `, we used a deterministic setting (temperature=0.0, top_p=1.0), while for making the final pruning result, we increased exploration (temperature=1.0, top_p=1.0). For Pattern Matching, the similarity threshold `latex $\tau$ ` was set to 0.6. Finally, during Surprisal-based Fine-grained Pruning, the maximum token budget was set to 4096 to ensure a deep level of compression.
