# Tab-CoT: Zero-shot Tabular Chain of Thought

**Authors:** Ziqi Jin and Wei Lu (StatNLP Research Group, Singapore University of Technology and Design)

**arXiv:** 2305.17812

**Code:** https://github.com/Xalp/Tab-CoT

---

## Abstract

The chain-of-though (CoT) prompting methods were successful in various natural language processing (NLP) tasks thanks to their ability to unveil the underlying complex reasoning processes. Such reasoning processes typically exhibit implicitly structured steps. Recent efforts also started investigating methods to encourage more explicitly structured reasoning procedures to be captured [Zhou et al., 2022]. In this work, we propose Tab-CoT, a novel tabular-format CoT prompting method, which allows the complex reasoning process to be explicitly modelled in a highly structured manner. Despite its simplicity, we show that our approach is capable of performing reasoning across multiple dimensions (i.e., both rows and columns). We demonstrate our approach's strong zero-shot and few-shot capabilities through extensive experiments on a range of reasoning tasks.

## Introduction

The chain-of-thought (CoT) prompting method [Wei et al., 2022] encourages the large language models (LLMs) to engage in a thought process before providing the answer to the given question. Such an approach shows impressive performance improvements in reasoning tasks. Notably, in the zero-shot setting, it was shown that a simple prompt such as "`let's think step by step`" could facilitate the step-by-step thinking process before answering the original question [Kojima et al., 2022]. Such a task-agnostic method unveiled that LLMs can be descent zero-shot reasoners.

[FIGURE: A comparison between Tab-CoT with standard prompting and zero-shot-CoT on the same question. Chain-of-thought prompts are highlighted in orange.]

The reasoning process is inherently structured. This gives rise to some new developments along this line of work recently. Specifically, Zhou et al. (2022) suggests an alternative prompting approach that enables a two-stage structured reasoning process. Gao et al. (2022) proposes an approach that involves code in the prompt design, allowing structured information in the form of formal language to participate in the reasoning process. While effective, such methods require specific prompt engineering for different domains or defining multiple variables, which can be difficult to maintain or keep track of.

Inspired by the fact that state-of-the-art large language models, such as GPT-3 [Brown et al., 2020] and CodeX [Chen et al., 2021], have the capability of reasoning over tabular structured data (because such models are trained on massive data collected from the Internet, which contains a large amount of tabular formed data), we propose a novel framework called _Tabular Chain of Thought_ (Tab-CoT) that models the structured reasoning process using a table-filling procedure.

We show that the model can perform step-by-step reasoning by creating a table without further fine-tuning by using a table header with column names in the form of "`|step|question|response|`" as a prompt. While conventional natural language texts are generated in a 1-dimensional sequential order, the table has a 2-dimensional structure, allowing inference along both columns and rows to be performed simultaneously. Unlike previous works which focused on extracting information from existing tabular structured data [Gong et al., 2020], our approach generates the table while performing the reasoning process (and extracts the answer from the generated table at the end).

[FIGURE: Overview of our zero-shot Tab-CoT method, which contains two steps: (1) table generation and (2) answer extraction. Added prompts are highlighted in orange. Texts generated by the LLM are highlighted in green.]

Our method generates a table as the output, which is more organized and concise than the output from the conventional CoT method. In this example, while zero-shot CoT generates 140 words, our method only generates 28. Besides, we found our method can reason horizontally and vertically at the same time. ("`74 loaves`" is the sum of "`68 loaves`" from the same row and "`6 loaves`" from the same column.) This demonstrates that our Tab-CoT method benefits from the 2-dimensional structure of the table, where the information can flow in two dimensions.

We summarize our main contributions in this work as follows:

- We propose a new approach called Tabular Chain-of-Thought (Tab-CoT) that utilizes a tabular structured reasoning scheme in combination with state-of-the-art large language models to generate answers. To the best of our knowledge, this is the first method that uses tables in a "chain of thought" process.

- The 2-dimensional tabular structure of Tab-CoT allows for improved unlocking of the step-by-step reasoning capabilities of LLMs, transforming the linear "chain of thought" process into a more structured one.

- Extensive experiments have revealed that our Tab-CoT outperforms traditional CoT techniques in zero and few-shot settings. This indicates that Tab-CoT has strong potential as a superior alternative to current chain-of-thought prompting methods.

## Related Work

Chain-of-thought prompting [Wei et al., 2022], a variation of few-shot prompting that adds step-by-step reasoning in those few-shot examples instead of just providing answers, has achieved significant improvements across multiple datasets. The LLMs can generate solutions following the solution format of prompts. Compared to traditional prompting, chain-of-thought prompting decomposes the task into smaller steps, which makes difficult tasks easier to solve.

The chain-of-thought prompting method is not necessarily purely natural language based. Program Aided Language Models (PAL) [Gao et al., 2022] provides few-shot samples that contain executable Python code. Such an approach enables the LLMs to interact with the Python shell, allowing the model to focus on learning how to do mathematical reasoning rather than numerical calculations.

These chain-of-thought methods provide the solution structure and pattern via few-shot samples, but can these be provided without these few-shot samples in the zero-shot setting? Zero-shot CoT [Kojima et al., 2022] is a zero-shot chain-of-thought prompting method. The prompt phrase "`Let's think step by step`" added after the question triggers the explicit reasoning process. However, compared to few-shot CoT [Wei et al., 2022], zero-shot CoT allows more flexibility in the structure of the reasoning process.

Recently, Zhou et al. (2022) proposed Least-to-Most prompting, which is a prompting strategy that reduces a complex problem into a list of sub-questions, and sequentially solves the sub-questions. Each sub-question is solved with the answer to previously solved sub-questions. Compared to zero-shot CoT, this method has more restrictions on the structure of reasoning by decomposing and sequentially answering. Moreover, importing external tools (like calculator and python shell) can further aid the math computation within the arithmetic domain [Gao et al., 2022].

These works reveal the importance of promoting structures in the chain-of-thought process. However, the nature of the zero-shot prompting makes the injection of structures into the generation process challenging. This motivates us to devise a better mechanism to prompt the language models under the zero-shot setting -- a new prompting scheme that allows highly structured outputs in the form of tables to be generated.

[FIGURE: Understanding how state-of-the-art LLM ("code-davinci-002") reason with tabular-structured data.]

## Tab-CoT

Similar to zero-shot CoT [Kojima et al., 2022], our method involves two prompts that can be used in large language models (LLMs), one for table generation and the other for answer extraction. While our method is primarily applied in zero-shot settings, it can also work in few-shot settings.

### Tables in LLMs

We found that in the official "parse unstructured data" demo provided by OpenAI (https://beta.openai.com/playground/p/default-parse-data), a table header is provided as part of the prompt, which is as follows: "`|Fruit|Color|Flavor|`". With such a prompt, the underlying LLM can automatically generate a table. This suggests possible formatting for tables in such state-of-the-art LLMs. And "|" is the recognizable delimiter of tables in OpenAI models.

To validate this observation, we queried the LLM "code-davinci-002" [Chen et al., 2021] with the following question: "`a=2, b=3, what is 2*a+3*b?`", and provided another table header: "`|step|solution|`" (temperature set to 0 for reproducibility). We found that it completes a structured table as follows:

```
a=2, b=3, what is 2*a+3*b?
|step|solution|
|:---|:---|
|1|2*a+3*b|
|2|2*2+3*3|
|3|4+9|
|4|13|
```

This experiment essentially unveils how the tables are represented in such LLMs. The results also illustrate how the table can potentially be used for generating a reasoning process. Next, to validate this, we designed several simple experiments to understand how reasoning over such tabular-structured data is performed on such LLMs. Our first experiment (A) shows that such LLMs are able to perform potential vertical reasoning. However, if we replace '`|`' with '`,`' (B), the LLM fails to capture the patterns in the data. This tells us that the correct formatting is crucial when reasoning with tables in such LLMs.

Next, we intentionally insert a mistake into the partial table and ask the model to continue the generation process (circled in C). Surprisingly, the LLM is able to generate the correct entries even though the mistake occurred in the same row. This further confirms the LLM's strong potential in performing vertical reasoning with tabular-structured data.

Moreover, to prove both vertical and horizontal reasoning exists, we increase the difficulty by directly appending the first two elements from step 9 after step 6 (D). If only vertical reasoning existed, the value under "`v4`" would have been "`11`". Instead, the value generated is "`13`," confirming that the LLMs have the potential to perform a combination of horizontal and vertical reasoning simultaneously.

### Table Generation Prompt

To make use of the 2-dimensional structure of the table, we replace the natural language prompt with a table-generation prompt (e.g., "`|step|question|response|`"), which serves as the header of the table. This regulates the context of this table, forcing the LLMs to conduct step-by-step reasoning by completing the table. Meanwhile, the choice of columns can be very specific. If each row of the table is regarded as a step, the row-by-row table generation process will become a step-by-step reasoning process. Within each step (row), we have multiple columns, each of which contributes certain detail towards the current reasoning step.

For any text question $x$, we have a table generation prompt (all column names) $c$. Concretely, we add the table generation prompt in the next row of the text question:

```latex
$$\textsf{LLM}(x,c) = \begin{bmatrix}
    c_{1} & c_{2} & \cdots & c_{n-1} & c_{n}\\
    t_{1,1} & t_{1,2} & \cdots & t_{1,n-1} & t_{1,n}\\
    \vdots & & \ddots & & \vdots\\
    t_{m,1} & t_{m,2} & \cdots & t_{m,n-1} & t_{m,n}
\end{bmatrix}$$
```

where $t_{1,1} \cdots t_{m,n}$ are the entries within the generated table, which contains $m$ rows and $n$ columns.

### Answer Extraction Prompt

After the table content, denoted as $T$, is generated from the previous step, we perform answer extraction. The answer extraction step helps us to extract the answer from the table, as the final results may not always be in the last cell of the generated table. Following zero-shot CoT [Kojima et al., 2022], we add another answer extraction prompt $a$: "`the answer is`" after the generated table, to extract the final answer from the table:

```latex
$$Answer = \textsf{LLM}(x,c,T,a)$$
```

### Structure-Promoting Table Scheme

Different table generation prompts (headers) may result in different tables generated (with different content). We propose a "structure-promoting scheme", which maximally unlocks the reasoning abilities of LLMs.

We define each row as a reasoning step. A table containing multiple rows will depict the step-by-step reasoning procedure leading to the final answer. Thus, our first column is "`step`", containing a number that indicates which reasoning step the current row represents.

Least-to-most prompting [Zhou et al., 2022] contains two stages: problem reduction and sequential solving. In problem reduction, they decompose a question into multiple subquestions. Similarly, we add "`subquestion`" as our second column. At the beginning of each step, the LLMs will generate a subquestion under this column, which demonstrates the objective of the current reasoning step.

The conventional zero-shot CoT [Kojima et al., 2022] shows that allowing the model to generate some reasoning process before answering can achieve a better result. Inspired by this observation, we add a third column, "`process`", into our table. Given a subquestion in the previous column, we expect to generate the reasoning process in the current column before answering.

The last column is named "`answer`". As the previous reasoning process under the "`process`" column may not necessarily provide an answer, we hope to use the "`answer`" column to explicitly request an (intermediate) answer at the end of each reasoning step.

With the above considerations, our primary scheme for the table header is designed as follows, which serves as our main table generation prompt:

```
|step|subquestion|process|result|
```

## Experimental Setup

### Large Language Models

We consider two state-of-the-art large language models under the GPT-3 family [Brown et al., 2020] in our experiments, namely "code-davinci-002" and "text-davinci-002", whose APIs are made available by OpenAI (https://openai.com/api/). For brevity, we use "code" to refer to the model "code-davinci-002" and "text" to refer to "text-davinci-002" in our experiments.

### Tasks and Datasets

We primarily focus on mathematical reasoning in this work. Thus, we evaluate our method on 6 arithmetic reasoning datasets. Specifically, they are SingleEq [Koncel-Kedziorski et al., 2015], AddSub [Hosseini et al., 2014], MultiArith [Roy and Roth, 2015], GSM8K [Cobbe et al., 2021], AQUA-RAT [Ling et al., 2017], and SVAMP [Patel et al., 2021], which are standard datasets widely used in the community.

We also conducted additional experiments on datasets involving other types of reasoning tasks. Specifically, we evaluate our method on two symbolic reasoning tasks: Last letter and Coin Flip (generated by Kojima et al., 2022): the former is the task that asks for the concatenation of the last letters of 4 words, and the latter asks for the state of the coin after being flipped a few times. We investigate how the specificity of column names affects the performance and report in our ablation study. We also evaluate our method on two commonsense reasoning tasks: CommonsenseQA [Talmor et al., 2019] and StrategyQA [Geva et al., 2021].

Following zero-shot CoT [Kojima et al., 2022], we set the first generated number as the numeral answer, the first capitalized letter as the answer for multiple-choice questions, and the first "yes" or "no" as the answer for "Yes or No" questions.

## Results

### Main Results

Our main experiments are conducted on arithmetic reasoning tasks under the zero-shot setting. We tested the performance of both text-based and code-based LLMs on all methods. Under the scheme "`|step|subquestion|process|result|`", our zero-shot Tab-CoT approach significantly outperformed the standard prompting in all tasks. Furthermore, our best-performing Tab-CoT model (using code-based LLM) outperforms the best conventional CoT model in 5 out of 6 tasks (with an average improvement of 2.2%).

**Zero-shot Results on Arithmetic Datasets:**

| Method             | LLM  | SingleEq | AddSub   | MultiArith | GSM8K    | AQUA     | SVAMP    | Average  |
| ------------------ | ---- | -------- | -------- | ---------- | -------- | -------- | -------- | -------- |
| Standard Prompting | text | 74.6     | 72.2     | 17.7       | 10.4     | 22.4     | 58.8     | 42.7     |
| Standard Prompting | code | 46.3     | 51.4     | 7.2        | 4.1      | 23.6     | 29.5     | 27.0     |
| CoT                | text | 78.0     | 69.6     | 78.7       | 40.7     | 33.5     | **62.1** | 60.4     |
| CoT                | code | 65.6     | 65.6     | 64.8       | 31.8     | 29.5     | 39.9     | 49.5     |
| Tab-CoT            | text | 74.6     | **71.9** | 72.2       | 39.3     | 36.6     | 57.0     | 58.6     |
| Tab-CoT            | code | **81.9** | 70.9     | **81.2**   | **44.4** | **37.0** | 60.5     | **62.6** |

When the standard prompting method is considered, using the text-based LLM leads to significantly better results than the code-based counterpart (15.7% on average). Similarly, when zero-shot CoT is considered, using the former also outperforms the latter by 10.9% on average. However, for our Tab-CoT approach, "code" outperforms "text" by 4.0%, leading to the best overall performance among all configurations.

From such results, we can see that the conventional CoT method responds differently from our Tab-CoT method with different types of underlying LLMs involved. The conventional CoT method (and the standard prompting method) strongly favors a text-based LLM under the zero-shot setting. In contrast, our approach works well with both types of LLMs, but the code-based version can give it an additional boost in performance. Compared with "text", the "code" model is further fine-tuned on code [Chen et al., 2021]. We conjecture that table generation resembles the code generation process -- both involve structured procedures that are highly organized and follow a step-by-step process. Comparing our Tab-CoT approach with conventional CoT, we can conclude that our proposed table-generation prompt is able to significantly better unlock the strong reasoning abilities within the code-based LLM.

Based on the above main experiments, we choose to use "code" as the default LLM for all subsequent experiments unless otherwise specified.

### Importance of Scheme Design

To understand the significance of our proposed table scheme design, we evaluate the performance of "`|step|subquestion|process|result|`", along with four variations, each of which is obtained by removing one of the four columns as ablation. The results show that each column of "`|step|subquestion|process|result|`" is crucial.

**Column Ablation Results (Average across datasets):**

| Scheme | Average     |
| ------ | ----------- | ----------- | ------- | ------ | ---- | -------- |
| `      | subquestion | process     | result  | `      | 54.3 |
| `      | step        | process     | result  | `      | 57.2 |
| `      | step        | subquestion | result  | `      | 61.3 |
| `      | step        | subquestion | process | `      | 60.9 |
| `      | step        | subquestion | process | result | `    | **62.6** |

From the result, we notice that removing the column "`step`" from our scheme results in the most significant performance drop. This implies although the step only contains a number indicating "which step this is", it organized the table in sequential order over rows. The column "`subquestion`" is also important. Removing "subquestion" from the scheme also shows an average performance drop of 5.4%. The "`subquestion`" column forms step-by-step instructions vertically, indicating the subquestion under consideration for each step. The "`step`" and "`subquestion`" columns play important roles in maintaining the structure of the table, building vertical connections across rows.

### Effectiveness of Self-Consistency

The self-consistency [Wang et al., 2022] decoding strategy was shown to obtain better results by generating and exploring multiple, diverse reasoning paths. We also adopt a similar approach here. In the original self-consistency paper, up to 40 reasoning paths were considered. We show the feasibility of using only 3 paths in our work (the self-consistency decoding method did not show significant improvement when the number of reasoning paths is below 5 in their paper). This is conveniently achieved by using 3 different prompts -- we select another two table schemes besides the standard scheme. One is a highly similar prompt, which we expect to perform similarly well, and the other is less similar, which we expect to yield a worse performance.

**Self-Consistency Results:**

| Scheme                         | SingleEq | AddSub      | MultiArith | GSM8K    | AQUA     | SVAMP    | Average  |
| ------------------------------ | -------- | ----------- | ---------- | -------- | -------- | -------- | -------- | ---- | ---- | ---- | ---- | ---- |
| `                              | step     | subquestion | process    | result   | `        | 81.9     | 70.9     | 81.2 | 44.4 | 37.0 | 60.5 | 62.6 |
| `                              | step     | subquestion | procedure  | result   | `        | 83.7     | 69.1     | 77.8 | 43.4 | 38.2 | 60.4 | 62.1 |
| `                              | step     | question    | response   | `        | 77.6     | 73.9     | 79.0     | 38.1 | 34.3 | 63.9 | 61.1 |
| Self-consistency (using above) | **86.4** | **78.2**    | **85.2**   | **48.2** | **44.1** | **66.9** | **68.2** |

We then perform majority voting based on the outputs from these 3 prompts. Interestingly, although a prompt with worse performance is used in the voting process, the overall performance improves. This shows the benefits of integrating different table schemes for such tasks, which helps improve the overall robustness of the approach.

### Few-shot Tab-CoT

Tab-CoT shows impressive reasoning ability under the zero-shot setting. It can generate a structured output in the form of a table that enables the chain-of-thought reasoning process without few-shot samples. Tables are capable chain-of-thought carriers, but can they also serve as good chain-of-thought teachers? To answer this question, we evaluated Tab-CoT under the few-shot setting.

For a fair comparison, we use the same few-shot sample questions described in Wei et al. (2022). We use "`|step|subquestion|process|result|`" as the table scheme when representing few-shot samples.

**Few-shot Results on Arithmetic Datasets:**

| Method             | SingleEq | AddSub   | MultiArith | GSM8K    | AQUA     | SVAMP    | Average  |
| ------------------ | -------- | -------- | ---------- | -------- | -------- | -------- | -------- |
| Standard Prompting | 86.8     | **90.9** | 44.0       | 19.7     | 29.5     | 69.9     | 68.2     |
| CoT                | **93.1** | 89.1     | 96.2       | **63.1** | 45.3     | 76.4     | 77.2     |
| Tab-CoT            | 92.1     | 89.1     | **96.3**   | 61.6     | **46.9** | **82.9** | **78.2** |

Our method outperformed few-shot CoT by 1% on average. While the performance difference between Tab-CoT and CoT on other datasets is below 2%, the performance difference on SVAMP is 6.5%. The large improvement on SVAMP is likely related to the selection of few-shot samples because Wei et al. (2022) select 8 sample questions from SVAMP for all arithmetic reasoning tasks except AQUA.

### Case Studies

The main experimental results show that "code" under-performs "text" with conventional CoT but yields better results in our Tab-CoT. To understand this better, we conduct case studies to compare their generated tables.

While "code" only generated short text snippets or formulas under "`process`", the words generated by "text" under the same column tend to form complete sentences whenever possible. As we mentioned earlier, "code" is an LLM that is further fine-tuned on code [Chen et al., 2021]. This explains why it appears more amenable to the tabular-structured format of the output. Overall, "code" shows better reasoning ability by demonstrating a more concise and straightforward reasoning process.

**Example Case Study:**

Question: _Gretchen has some coins. There are 30 more gold coins than silver coins. If she had 70 gold coins, how many coins did Gretchen have in total?_ (Ground Truth: **110**)

CoT output: Let's think step by step. If Gretchen had 70 gold coins, then she would have 30 silver coins (70 - 30 = 40). So Gretchen would have a total of **100** coins (70 + 30 = 100). [INCORRECT]

Tab-CoT (code) output:
| step | subquestion | process | result |
|------|-------------|---------|--------|
| 1 | How many gold coins does Gretchen have? | 30 more gold coins than silver coins | 70 gold coins |
| 2 | How many silver coins does Gretchen have? | 30 more gold coins than silver coins | 40 silver coins |
| 3 | How many coins does Gretchen have in total? | 70 gold coins + 40 silver coins | **110** coins |

[CORRECT]

[FIGURE: The schemes that disable (left) and enable (right) potential 2-dimensional reasoning.]

### Additional Experiments

We further evaluate our methods on symbolic reasoning and commonsense reasoning tasks. We also conducted some new experiments based on the GPT-3.5 model to understand our approach's effectiveness on such newer models (GPT-3.5 was released in March 2023).

#### Symbolic Reasoning

We evaluate Tab-CoT on two symbolic reasoning datasets: Coin Flip (CF) and Last Letter (LL). Unlike the arithmetic reasoning tasks, these tasks focus on some specific problems. This also opens up the opportunity for us to examine whether the specificity of the table scheme may have an impact on the reasoning process in such tasks.

To this end, we split table schemes into three categories: (1) _general_: the table scheme that can be generally applied to most text questions. (2) _domain-specific_: the table scheme that can be adapted to a specific domain. (3) _task-specific_: the scheme that can only be adopted by a single task.

**Symbolic Reasoning Results:**

| Task | Category    | Prompt                     | Result |
| ---- | ----------- | -------------------------- | ------ | --------------- | ----------- | -------------- | --- | -------- |
| CF   | -           | `let's think step by step` | 91.4   |
| CF   | 1 (general) | `                          | step   | subquestion     | process     | result         | `   | 85.0     |
| CF   | 2 (domain)  | `                          | step   | initial state   | action      | next state     | `   | 80.4     |
| CF   | 3 (task)    | `                          | step   | name            | flip or not | result         | `   | **96.2** |
| LL   | -           | `let's think step by step` | 57.6   |
| LL   | 1 (general) | `                          | step   | subquestion     | process     | result         | `   | 25.2     |
| LL   | 2 (domain)  | `                          | step   | original answer | action      | updated answer | `   | 50.8     |
| LL   | 3 (task)    | `                          | step   | word            | last letter | answer         | `   | **72.8** |

Our experiments illustrate that the specificity of the table schemes highly affects the performance of symbolic reasoning tasks. One may expect the performance to increase as the table scheme becomes more task-specific. Our task-specific scheme outperformed the zero-shot CoT in both tasks. However, the increased specificity does not always lead to higher accuracy. In the Coin Flip task, we noticed that another task-specific scheme "`|step|initial coin state|flip or not|next coin state|`" only achieves an accuracy of 68.0%. Although that scheme is more task-specific, it largely disabled the vertical reasoning in the table. The general scheme effectively enables reasoning along both vertical and horizontal directions, leading to significantly better results.

#### Commonsense Reasoning

As another set of additional experiments, we further evaluate our method on commonsense reasoning, including CommonsenseQA [Talmor et al., 2019] and StrategyQA [Geva et al., 2021].

**Commonsense Reasoning Results (Zero-shot):**

| Method             | CommonsenseQA | StrategyQA | Average  |
| ------------------ | ------------- | ---------- | -------- |
| Standard Prompting | 69.0          | 3.3        | 36.2     |
| CoT                | 54.6          | 38.9       | 46.8     |
| Tab-CoT            | 68.4          | 50.4       | **59.4** |

Tab-CoT obtained the highest average accuracy. However, the results of our method did not show significantly improved performance compared with Standard Prompting in a few-shot setting. These results imply that commonsense reasoning tasks do not have a fixed answering pattern. Therefore, providing chain-of-thought samples is not enough to make up for the lack of commonsense knowledge.

#### Results on GPT-3.5

We test our method on the recent model "GPT-3.5-turbo-0301".

**GPT-3.5 Results (Zero-shot):**

| Method  | SingleEq | AddSub   | MultiArith | GSM8K    | AQUA     | SVAMP    | Average  |
| ------- | -------- | -------- | ---------- | -------- | -------- | -------- | -------- |
| CoT     | 85.6     | 83.3     | **90.5**   | 68.7     | 50.8     | 79.0     | 76.3     |
| Tab-CoT | **87.8** | **85.8** | 89.3       | **78.2** | **51.2** | **81.1** | **78.9** |

We found that our method is applicable to GPT-3.5, and achieves better performance compared to conventional Zero-shot CoT. Another interesting observation is when prompting the GPT-3.5 model with "Let's think step by step", a large number of the generated texts already contain a table in their CoT process (though those tables appear to be mostly used to organize information related to the question but do not appear to be used for presenting reasoning steps).

### Ablation Studies

#### Model Sizes

Kojima et al. (2022) evaluated the family of GPT-3 models of four different sizes: 2.7B, 6.7B, 13B, and 175B parameters. The results show that only the largest model ("text-davinci-002") shows the chain-of-thought reasoning ability.

We compare the performance of the smaller model "code-cushman-001" (13B) with "code-davinci-002" (175B). Similar to zero-shot CoT, smaller models do not show the ability to conduct chain-of-thought reasoning. The performance of "code-cushman-001" cannot reach 10%, except AQUA (a multiple choice dataset with 5 choices for each question).

| Task        | code-cushman-001 (13B) | code-davinci-002 (175B) |
| ----------- | ---------------------- | ----------------------- |
| SingleEq    | 6.3                    | 81.9                    |
| AddSub      | 6.3                    | 70.9                    |
| MultiArith  | 2.0                    | 81.2                    |
| GSM8K       | 0.9                    | 44.4                    |
| AQUA        | 16.9                   | 37.0                    |
| SVAMP       | 5.0                    | 60.5                    |
| **Average** | **6.2**                | **62.6**                |

#### Structure-Promoting Scheme

Results suggest that each column of our proposed scheme is important because removing any column will lead to a drop in performance.

## Discussion

Our experimental results confirmed the effectiveness of our proposed tabular chain-of-thought method under both zero-shot and few-shot settings. We summarize several advantages of our method compared to conventional chain-of-thought methods and list them below.

Tab-CoT generates a table illustrating the reasoning process, which is more _organized_. This nature of the generated text makes the reasoning process much easier.

Additionally, we conclude that Tab-CoT encourages a more _structured_ reasoning process to be explicitly modelled. As a 2-dimensional data structure, tables enable both horizontal reasoning along rows and vertical reasoning along columns.

Practically, table schemes are also _easy to craft_. Designing a specific table generation prompt typically involves deciding concise header names without concerning grammar. It is thus less cumbersome than choosing a natural language prompt from a diverse set of candidates.

Overall, we argue that under current state-of-the-art LLMs, table schemes are _natural prompts_ that are well suited for zero-shot learning.

## Conclusion

In this paper, we propose Tab-CoT, a novel prompting framework that performs effective zero-shot reasoning by generating a table.

Tab-CoT shows competitive results on arithmetic reasoning tasks under both zero-shot and few-shot settings. We further conducted comprehensive experiments across different reasoning tasks under different settings. Our comprehensive experiments revealed some specific benefits of our method and identify the optimal way to use it. We hope that, through our work, we can sparkle new ideas and provide some inspiration to our community.

In the future, we would like to explore methods to automate the scheme selection process, using the generated schemes to meet task-specific requirements. Future work also includes integrating external calculators [Gao et al., 2022], or task-specific supervision [Zhou et al., 2022] into the learning process, under both zero-shot and few-shot settings.

Our Tab-CoT also provides a straightforward decomposition of the intermediate thought process. This highly structured chain of thought produced by our approach may help people to observe and interpret how large language models decompose complex problems. We believe our proposed method can help reveal the underlying mechanisms associated with the emergence of certain complex behaviours associated with large language models.

## Limitations

We identify a few limitations of this work. First, our approach is applicable to language models pre-trained with tables, which may not always be included in all language models (especially small ones). Second, our approach's limited improvement in commonsense reasoning tasks suggests that its effectiveness may depend on the specific task and the level of structured reasoning required.

---

## Appendix: One-shot Reasoning on Symbolic Reasoning

We evaluate our method on Coin Flip and Last Letter under the one-shot setting. By adding one few-shot sample, LLMs can gain a significant performance boost in both tasks with general scheme "`|step|subquestion|process|result|`".

| Task | Prompt | Zero-shot | One-shot    |
| ---- | ------ | --------- | ----------- | ------- | ------ | --- | ---- | ----- |
| CF   | `      | step      | subquestion | process | result | `   | 85.0 | 100.0 |
| LL   | `      | step      | subquestion | process | result | `   | 25.2 | 96.0  |

## Appendix: Few-Shot Samples

**Example few-shot sample for arithmetic reasoning:**

**Q:** There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done, there will be 21 trees. How many trees did the grove workers plant today?

| step | subquestion                                                     | process | result |
| ---- | --------------------------------------------------------------- | ------- | ------ |
| 1    | How many trees are in the grove?                                | 15      | 15     |
| 2    | How many trees will be in the grove after the workers are done? | 21      | 21     |
| 3    | How many trees did the workers plant?                           | 21 - 15 | 6      |

Therefore, the answer (arabic numerals) is 6.
