How to Use Self-Consistency Prompting

Self-consistency is a decoding strategy layered on top of chain-of-thought. Instead of taking one reasoning chain produced with greedy decoding, you sample several independent chains for the exact same question, then marginalize over the reasoning and keep only the final answers — choosing whichever answer appears most often. The reasoning is a means to an end; the vote is on the conclusion.

The intuition from Wang et al. 2022 is that complex problems admit multiple valid reasoning paths that should all land on the same correct answer, whereas flawed paths tend to produce a spread of different wrong answers. Aggregating across samples therefore amplifies the signal (the convergent correct answer) and averages out the noise.

It is sometimes called 'sample-and-vote' or 'sample-and-marginalize.' Crucially, it only works when the final answer is something you can compare and count — a number, a label, a choice — so that 'most common' is well defined.

Self-consistency earns its cost on **hard, multi-step problems with a single verifiable answer**: arithmetic and math word problems, logical deduction, structured classification, and commonsense questions where a single greedy pass is unreliable. These are exactly the tasks Wang et al. 2022 studied, all of which benefit from chain-of-thought first.

It is **overkill** when a single pass is already reliable, when the task is a simple lookup, or when the output is open-ended (essays, creative writing, long plans) where there is no discrete answer to vote on. You cannot majority-vote five different paragraphs.

There is also a 2026 wrinkle. Frontier reasoning models — GPT-5.5 in thinking mode, Claude Opus 4.8 and Sonnet 4.6 with extended thinking — already do extensive internal reasoning and are far more reliable per single pass than the 2022-era models the technique was designed for. On those models, reach for self-consistency only on genuinely hard problems where a single pass still wavers; on routine work it mostly multiplies your cost. Check current model positioning in how to choose an AI model.

Use self-consistency when: the task is a hard, multi-step problem with one verifiable answer (math, logic, classification), a single greedy pass is unreliable, and the extra token cost is worth the accuracy. It pairs with chain-of-thought, not as a replacement for it.
Skip self-consistency when: a single pass is already reliable, the task is a simple lookup, or the output is open-ended (essays, plans, creative writing) with no discrete answer to count. On frontier reasoning models, reserve it for genuinely hard cases — otherwise it just multiplies cost.

Start with a single chain-of-thought pass — the 'before':

``` A train leaves at 2:15 PM going 60 mph. A second train leaves the same station at 2:45 PM going 75 mph on the same track in the same direction. At what time does the second train catch the first? Reason step by step, then give the answer. ```

Run once, and a fast model may make an arithmetic slip and report a confident but wrong time. The self-consistency 'after' keeps the same prompt but changes how you run it:

``` Run the SAME prompt below 5 times, each with temperature ~0.7 (sampling on), as 5 independent calls. Prompt: "A train leaves at 2:15 PM going 60 mph. A second train leaves the same station at 2:45 PM going 75 mph on the same track in the same direction. At what time does the second train catch the first? Reason step by step, then end with a line: ANSWER: <time>." Then collect the 5 ANSWER lines and report the time that appears most often, plus the count (e.g., 'ANSWER: 4:45 PM — 4 of 5 runs agreed'). ```

Two things make this work. First, sampling is on (temperature above 0) so the five runs explore different reasoning paths instead of producing identical greedy output. Second, the pinned `ANSWER:` line makes the final answers trivial to extract and count.

---

You can run the votes manually (paste the prompt five times) or programmatically: loop five API calls with temperature set, parse the `ANSWER:` line from each, and `mode()` the results. The programmatic version is what production systems use, and it's where the cost trade-off is explicit — five calls instead of one.

Self-consistency multiplies cost and latency by the number of samples: five samples is roughly five times the tokens of a single CoT pass, since each run regenerates the full reasoning. There is no free lunch — you are buying reliability with compute. Estimate the hit against live rates on the OpenAI pricing page, Anthropic pricing page, or Gemini pricing page, and compare model costs in cost per token, all major models.

More samples generally help up to a point and then plateau — each additional vote adds less. A small odd number (so ties are rare) is the usual starting point; raise it only if the answers are still splitting. Tune empirically on your own task rather than chasing a fixed number.

Because cost scales with samples, self-consistency is best deployed selectively: trigger it only for the hard subset of inputs, and use a single pass for the easy majority. That keeps the accuracy where it matters without paying the multiplier on every request.

Self-consistency sits on top of chain-of-thought: CoT produces each reasoning path, and self-consistency aggregates across many of them. It differs from Tree of Thoughts (arXiv:2305.10601), which actively explores and prunes a branching search of reasoning steps rather than independently sampling whole chains and voting at the end.

It also composes with role prompting (sample several expert opinions, then vote) and with least-to-most prompting (decompose first, then self-consistency on the hardest subproblem). For the full landscape of variants, see the chain-of-thought variants comparison and the DAIR.ai Prompt Engineering Guide.