# Abstract

Self-ensembling techniques with diverse reasoning paths such as Self-Consistency have demonstrated remarkable performance gains in text generation with Large Language Models (LLMs). However, such techniques depend on the availability of an accurate answer extraction process to aggregate across multiple outputs. Moreover, they acquire higher inference cost, in comparison to Greedy Decoding, due to generation of relatively higher number of output tokens. Research has shown that the free form text outputs from Self-Consistency can be aggregated reliably using LLMs to produce the final output. Additionally, recent advancements in LLM inference have demonstrated that usage of diverse exemplars in prompts have the ability to induce diversity in the LLM outputs. Such proven techniques can be easily extended to self-ensembling based approaches to achieve enhanced results in text generation. In this paper, we introduce PEDAL (Prompts based on Exemplar Diversity Aggregated using LLMs), a hybrid self-ensembling approach, that combines the strengths of diverse exemplar based prompts and LLM based aggregation to achieve improvement in overall performance. On the publicly available SVAMP and ARC datasets, our experiments reveal that PEDAL can achieve better accuracy than Greedy Decoding based strategies with lower inference cost compared to Self Consistency based approaches.

# Introduction

Large Language Models (LLMs) [brown2020GPT3; Raffel2020T5; Chowdhery2022PaLMSL; Touvron2023LLaMAOA] have been proven to show remarkable performance in a wide range of Natural Language Understanding tasks [zhao2023surveylargelanguagemodels] as a result of their outstanding reasoning capabilities [Wei2022ChainOT; Zhou2022LeasttoMostPE]. However, they still rely on carefully designed prompts to achieve optimal performance [khattab2023dspycompilingdeclarativelanguage; fernando2023promptbreederselfreferentialselfimprovementprompt]. To realize further improvement in LLM reasoning, [Wang2022SelfConsistencyIC] proposed a self-ensembling technique termed "Self-Consistency"(SC) where diverse "Chain-of-Thought"(CoT) [Wei2022ChainOT] reasoning paths were generated and then aggregated to construct an accurate and reliable response. This approach has been successfully extended to various use-cases such as LLM hallucination detection [chen2024insidellmsinternalstates], medicine [zhou2024LLMSurveyMedocine] and code generation [huang2024enhancinglargelanguagemodels].

While SC based approaches can significantly improve the robustness of LLM outputs, one of their common drawbacks is that they perform best on a fixed answer set [Wang2022SelfConsistencyIC] or rely on training custom aggregation methods to measure consistency across multiple text outputs. To address this, [Chen2023UniversalSF] proposed "Universal Self Consistency"(USC), an extension of SC, that aggregated the text outputs by re-invoking the LLM. Essentially, USC prompted the LLM to select the most consistent response among the different candidate answers generated by SC and demonstrated that it can achieve improved performance. However, this still leaves us with another drawback of SC which is the cost involved in generating the outputs. Concretely, SC involves generating long and diverse reasoning paths which results in a higher number of output tokens compared to Greedy Decoding based approaches. The cost of output token generation with LLMs is typically more than input token processing due to the difference in the number of forward passes [shazeer2019fasttransformerdecodingwritehead; Chng2024OutputTokenGenCost] resulting in a higher inference cost with SC.

[li-etal-2023-diverse-prompt] experimented with usage of diverse exemplars in the LLM prompts and combined them with diverse reasoning paths in SC to achieve more accurate results in text generation. We observe that if we leverage diverse exemplars with Greedy Decoding for text generation and aggregate the responses as in USC, we achieve better performance than traditional Greedy Decoding in terms of accuracy while also achieving lower cost of inference in comparison to SC based approaches.

In this paper, we present a hybrid self-ensembling approach, PEDAL (**P**rompts based on **E**xemplar **D**iversity **A**ggregated using an **L**LM), that offers a trade-off between the Greedy Decoding and SC in terms of accuracy and cost efficiency. We leverage diverse exemplars in LLM prompts to generate multiple candidate responses using Greedy Decoding and then aggregate them using an LLM to generate the final response. On two publicly available datasets, we demonstrate that PEDAL achieves better accuracy than Greedy Decoding based strategies and offers lower cost in inference compared to SC based strategies.

Rest of the paper is organized as follows: In Section 2, we describe previous work for solving similar problems. Section 3 explains our proposed strategy in detail followed by Section 4 where we describe the data and the experiment settings to validate PEDAL. We then present our results and analyses in Section 5. Finally, in Section 6, we summarize our findings and discuss potential future work.

# Related Work

LLMs have been widely studied and applied in a variety of tasks including code generation [zheng2024surveylargelanguagemodelsCode], finance [li2024largelanguagemodelsfinance], law [yu2022legalpromptingteachinglanguage] and so on. However, none of the LLMs seem to consistently outperform the rest of the models across all tasks [jiang2023llmblenderensemblinglargelanguage]. This led to exploring ensembling approaches with LLMs. Research focused on Prompt Chaining [Chase_LangChain_2022], Fusion [li2023deepmodelfusionsurvey], Mixture of Experts [cai2024surveymixtureexperts] and many more have shown promising results in combining LLMs to enhance the overall performance.

## Self Ensembling Strategies

[long2023largelanguagemodelguided; yao2023treethoughtsdeliberateproblem] generalized CoT to organize language model generated "thoughts" into a tree structure for solution search. However, similar to [Wang2022SelfConsistencyIC], they rely on custom aggregation methods to construct the final output. [Chen2023UniversalSF] addressed this issue by leveraging LLMs to perform majority consensus based aggregation without any specific model fine-tuning. In our work, we leverage a similar strategy to aggregate multiple candidates with a focus on the impact of using diverse LLM prompts as opposed to diverse reasoning paths.

## Prompt Ensembling Strategies

With the advent of LLMs, lot of research focused on developing effective prompting techniques [bach-etal-2022; lu-etal-2022-fantastically-pe] that have been extended by multiple prompt ensembling techniques [zhang2023preferpromptensemblelearning; pitis2023boostedpromptensembleslarge] to achieve further improvement. [singh-etal-2023-tree] built a decision tree of prompts that links multiple LM calls to solve a task. [arora2022askanythingsimplestrategy] used multiple prompt templates to reformat few-shot example inputs into an open ended question-answering format and then leverage Weak Supervision [WeakSupervisionSnorkel2017] to aggregate the LLM predictions. [Hou2023PromptBoosting] applied AdaBoost [schapire2013explaining] algorithm over a pre-defined prompt set for text classification by pairing prompts with the corresponding output distribution to construct a large pool of weak learners. [li-etal-2023-diverse-prompt] enhanced SC with diverse prompts by randomly selecting different exemplars for prompt construction, followed by sampling reasoning paths for each such prompt and then scoring the quality of each reasoning path using a custom trained model. While our work also leverages a similar prompt construction strategy, we aggregate the predictions without relying on explicitly training a task-specific model. Additionally, we focus on leveraging such prompt based strategies to reduce LLM inference cost rather than enhancing SC based approaches.

[IMAGE: High level overview of PEDAL (Prompts based on Exemplar Diversity Aggregated using an LLM)]

## LLM Inference Cost

To solve the problem of inference cost, researchers have commonly explored model compression techniques [zhu2024surveymodelcompressionlarge] such as model quantization [Jacob_2018_CVPR], model pruning [cheng2024surveydeepneuralnetwork] and model distillation [Gou_2021] aimed at reducing the size of the model without hurting the performance significantly. [shazeer2019fasttransformerdecodingwritehead] proposed sharing keys and values across all of the different attention heads in the transformer architecture, thus, reducing the memory bandwidth requirements of incremental decoding. [wu2024paralleldecodinghiddentransfer] explored decoding multiple successive tokens simultaneously in a single forward pass to reduce the inference time. FrugalGPT [chen2023frugalgptuselargelanguage] proposed a cascade of LMs that stops when an intermediate output is considered reliable, resulting in better computational efficiency. In our work, we focus on reducing the number of output tokens during LLM inference in comparison to SC while achieving better accuracy than Greedy Decoding.

# Methodology

Figure 1 shows the high level overview of our proposed system. The LLM generates multiple candidate responses using Greedy Decoding with prompts based on diverse exemplars. The candidate responses are then aggregated using the same LLM to generate the final output.

## Prompts with Diverse Exemplars

Traditional CoT based approaches rely on a single prompt comprised of a fixed set of exemplars. [li-etal-2023-diverse-prompt] showed that constructing multiple prompts, by modifying the exemplars chosen for the purpose of In-Context-Learning (ICL), further enhances the reasoning capability of language models. On similar lines, we construct multiple LLM prompts by randomly sampling the exemplars for ICL multiple times using different seed settings. For each such LLM prompt, we generate a candidate response using Greedy Decoding.

## LLM-based Aggregation

USC [Chen2023UniversalSF] that has been shown to accurately select the most consistent response among multiple SC responses using majority consensus. We follow USC and extract the final response from multiple candidate responses accordingly.

# Experiments

## Dataset

We consider two publicly available datasets for the purpose of our experiments -

- **SVAMP** [patel-etal-2021] Comprises of elementary-level Math Word Problems. Each problem consists of a short natural language narrative that describes a state of the world and poses a question about some unknown quantities.

- **AI2 Reasoning Challenge (ARC)** [clark2018thinksolvedquestionanswering] is a multiple-choice question-answering dataset, containing questions from science exams from grade 3 to grade 9 and is further split in two partitions - 'ARC-Easy' and 'ARC-Challenge' where 'ARC-Challenge' partition contains relatively more difficult questions that require reasoning

We report results on the validation split of each dataset. We restrict the ARC dataset to 'ARC-Challenge' only and work with 30% of the data sampled at random. Table 1 captures the corresponding details of the validation datasets considered for the experiments in the paper.

**Table 1: Validation dataset size for SVAMP and ARC datasets**

| Dataset Name | Number of Validation Samples |
| ------------ | ---------------------------- |
| SVAMP        | 300                          |
| ARC          | 345                          |

## Baseline Strategies

To benchmark our approach, PEDAL, we include the following baselines

- **Greedy Decoding** - We run the LLM to select the token with the highest probability at each step to generate the final output.

- **USC** - We run SC with CoT prompting and select the most consistent answer among all candidate responses using the same LLM.

- **Unified Diverse Exemplars** - To understand the impact of multiple candidate responses generated in PEDAL using diverse prompts, we combine all such diverse exemplars directly into a single ICL prompt and run Greedy Decoding. We refer to this baseline as "Unified Diverse Exemplars" (UDE).

## Experiment Setting

Each of the strategies were run using Qwen2-7B-Instruct [yang2024qwen2technicalreport] and Llama-3-8B-Instruct [Touvron2023LLaMAOA]. We measure the performance using accuracy and the number of output tokens. For purposes of reporting, we also share the number of input tokens consumed by the strategies. The LLMs were run using 4-bit quantization [dettmers2023qloraefficientfinetuningquantized]. Each experiment is run under three random seed settings for reproducibility. We pick three exemplars per experiment for the ICL prompt construction with each dataset. For each experiment, USC is run to generate three intermediate outputs and PEDAL is run with three diverse input prompts.

**Table 2: Performance comparison of Greedy Decoding, USC, UDE and PEDAL for SVAMP dataset using Accuracy**

| Model  | Approach  | Accuracy               |
| ------ | --------- | ---------------------- |
| Qwen2  | Greedy    | 76.0 +/- 1.52          |
|        | **USC**   | **80.33** +/- **0.98** |
|        | UDE       | 75.67 +/- 0.0          |
|        | PEDAL     | 77.89 +/- 1.28         |
| Llama3 | Greedy    | 70.22 +/- 1.03         |
|        | USC       | 72.99 +/- 0.47         |
|        | UDE       | 70.67 +/- 0.0          |
|        | **PEDAL** | **74.11** +/- **0.57** |

Averaged scores across 3 seeds are reported along with the standard deviation. Best performing strategy per model has been highlighted in bold.

**Table 3: Performance comparison of USC and PEDAL for SVAMP dataset using the number of output tokens**

| Model  | Approach  | Input             | Output                  |
| ------ | --------- | ----------------- | ----------------------- |
| Qwen2  | USC       | 902.89 +/- 2.16   | 502.75 +/- 1.43         |
|        | **PEDAL** | 1342.18 +/- 86.87 | **191.99** +/- **0.22** |
| Llama3 | USC       | 693.46 +/- 8.79   | 923.56 +/- 1.51         |
|        | **PEDAL** | 1261.51 +/- 64.95 | **197.72** +/- **0.2**  |

Averaged counts across 3 seeds are reported along with the standard deviation. Best performing strategy per model has been highlighted in bold.

**Table 4: Performance comparison of greedy decoding, USC, UDE and PEDAL for ARC dataset using Accuracy**

| Model  | Approach  | Accuracy               |
| ------ | --------- | ---------------------- |
| Qwen2  | Greedy    | 83.38 +/- 0.55         |
|        | **USC**   | **84.35** +/- **0.62** |
|        | UDE       | 84.06 +/- 0.0          |
|        | PEDAL     | 83.77 +/- 0.47         |
| Llama3 | Greedy    | 76.52 +/- 1.44         |
|        | USC       | 71.88 +/- 0.71         |
|        | UDE       | 76.52 +/- 0.0          |
|        | **PEDAL** | **78.55** +/- **0.47** |

Averaged scores across 3 seeds are reported along with the standard deviation. Best performing strategy per model has been highlighted in bold.

**Table 5: Performance comparison of USC and PEDAL for ARC dataset using the number of output tokens**

| Model  | Approach  | Input              | Output                  |
| ------ | --------- | ------------------ | ----------------------- |
| Qwen2  | USC       | 1153.04 +/- 1.96   | 668.71 +/- 7.19         |
|        | **PEDAL** | 1179.76 +/- 100.10 | **99.47** +/- **10.05** |
| Llama3 | USC       | 1072.96 +/- 5.67   | 928.1 +/- 1.31          |
|        | **PEDAL** | 1185.27 +/- 115.08 | **196.83** +/- **0.11** |

Averaged counts across 3 seeds are reported along with the standard deviation. Best performing strategy per model has been highlighted in bold.

# Results and Analysis

**Table 6: Effect of number of prompts on performance using Qwen2 with SVAMP and ARC datasets**

| Number of Prompts | SVAMP          | ARC            |
| ----------------- | -------------- | -------------- |
| 2                 | 77.0 +/- 0.98  | 83.96 +/- 0.36 |
| 3                 | 77.89 +/- 1.28 | 83.77 +/- 0.47 |
| 4                 | 78.22 +/- 1.34 | 83.87 +/- 0.49 |

Averaged scores across 3 seeds are reported along with the standard deviation.

Table 2 and Table 3 show the performance metrics for different strategies using SVAMP dataset. Similarly, Table 4 and Table 5 capture the performance metrics for the ARC dataset. We observe that our proposed approach consistently performs better than Greedy Decoding in terms of accuracy and outperforms USC in terms of the number of output tokens.

## Arithmetic Reasoning

As shown in Table 2, PEDAL displays improvement over Greedy Decoding on the SVAMP dataset. With Qwen2, PEDAL achieves an average accuracy of 77.89% while Greedy Decoding achieves an average accuracy of 76% implying a 1.89% improvement. PEDAL also outperforms UDE which achieves an accuracy of 75.67%. USC achieves the accuracy of 80.33%. Similarly, with Llama3, we observe that PEDAL achieves an average accuracy of 74.11% while Greedy Decoding achieves a score of 70.22% resulting in 3.89% improvement. However, with Llama3, we observe that USC achieves an accuracy of 72.99% which is lesser than PEDAL while UDE achieves an accuracy 70.67% marginally outperforming Greedy Decoding.

As shown in Table 3, with Qwen2, USC processes approximately 903 input tokens and 503 output tokens while PEDAL processes 1,343 input tokens with 192 output tokens making our approach evidently more cost efficient. With Llama3, USC processes an average of 694 input tokens and 924 output tokens while PEDAL processes 1,262 input tokens and 198 output tokens. While USC relies on lesser input tokens than PEDAL, the cost of output tokens with USC is more than 4 times the output token cost with PEDAL making our approach more cost efficient.

## Multiple-Choice Question Answering

As shown in Table 4, the strategies show a similar relationship with experiments run on the ARC dataset. With Qwen2, PEDAL achieves a marginal improvement of 0.39% over Greedy Decoding with an average accuracy of 83.77% while Greedy Decoding has an average accuracy of 83.38%. UDE outperforms PEDAL with an accuracy of 84.06% while USC still achieves the best performance with an accuracy of 84.35%. With Llama-3, PEDAL shows a 2.03% improvement with a score of 78.55% and greedy decoding achieves 76.52%. UDE achieves an accuracy of 76.52% matching the performance of Greedy Decoding. Surprisingly, USC achieves an accuracy of 71.88% which is relatively the least among the strategies. With USC, the main goal of the paper is to benchmark the proposed approach in terms of token count. To prevent diverging from the primary focus area, we leave deeper analysis of this behaviour to future work.

As shown in Table 5, with Qwen2, our approach outperforms USC where USC processes roughly 1,154 input tokens and 669 output tokens on an average while PEDAL processes 1,180 input tokens with 100 output tokens. With Llama3, USC processes 1,073 input tokens and 929 output tokens while PEDAL processes 1,186 input tokens and 197 output tokens. Our approach is the better choice in terms of the number of output tokens processed by the LLM.

## Comparison to CoT

Similar to PEDAL, CoT has been shown to be more accurate than Greedy Decoding and less expensive in terms of inference compared to SC. Based on pre-liminary interpolation of the number of output tokens using Table 3 and Table 5, we compare the number of output tokens consumed in a single intermediate output in SC (equivalent to CoT) with the number of output tokens in PEDAL. With Llama3, we observe that PEDAL would be more cost efficient for both datasets. With Qwen2, we observe that PEDAL would be more cost efficient for the ARC dataset but may prove to be more expensive for the SVAMP dataset in comparison to CoT. While PEDAL seems to be more reliably consistent, it would be interesting to further investigate and arrive at definitive conclusions. We intend to evaluate the merits and drawbacks of both approaches in a practical setting in future work.

## Impact of Number of Diverse Prompts

We re-run the experiments for both datasets with our best performing model, Qwen2, by varying the number of prompts to study how it affects the performance. As shown in Table 6, we additionally run the experiments for two and four diverse prompts under three seed settings. We observe slight improvements as we increase the number of prompts with the SVAMP dataset. However, we do not observe any such specific pattern with the ARC dataset.

# Conclusion

In this paper, we explored self-ensembling with LLMs using diverse exemplars with LLM based output aggregation. We observed that this combination can perform better than Greedy Decoding in terms of accuracy and achieve better cost efficiency than SC based methods. However, we restricted the experiments to small datasets that allowed benchmarking approaches using exact match without additional manual annotation efforts. In future work, we plan to explore possibilities on extending such ensembling strategies to a wider range of problem settings involving free-form text generation to further deep dive into strengths and weaknesses of our proposed system.
