Chain-of-Thought Variants in 2026: Zero-Shot, Few-Shot, Self-Consistency, Tree of Thoughts — and When Each Wins

**Mechanic:** Append 'Let's think step by step' (or equivalent) to the prompt. Model produces reasoning before final answer. Per Kojima et al. 2022 (arXiv:2205.11916), this single phrase unlocks substantial reasoning ability with no examples.

**Cost:** 1 model call, ~1.5-2× the output tokens of non-CoT (reasoning text adds to output length).

**Accuracy lift:** +15-30% on math/logic tasks vs. direct prompting in the original Kojima paper. Modern frontier models internalize this technique partly; lift is smaller on the latest models (~5-15%) because they default to reasoning when the task seems to need it.

**Best for:** Math problems, logic puzzles, multi-step reasoning tasks where the model needs to think through intermediate steps. Cheapest CoT variant.

**Mechanic:** Include 2-5 worked examples in the prompt that show the reasoning chain explicitly. The model imitates the demonstrated reasoning pattern.

**Cost:** 1 model call, but with significantly larger input (examples add 1-3K tokens typically). Per-query cost ~1.5-3× zero-shot CoT.

**Accuracy lift:** +20-40% over direct prompting on the original Wei et al. 2022 benchmarks. Improvement over zero-shot CoT is ~5-15% on most tasks; larger on niche/unusual reasoning patterns.

**Best for:** Tasks where the reasoning chain has domain-specific structure (legal reasoning, mathematical proofs, scientific analysis) and worked examples teach the model the expected pattern. Use 3-5 examples; more than 5 produces diminishing returns.

**Mechanic:** Generate N reasoning chains via temperature-sampled multiple calls. Take the majority-vote answer across chains. Per Wang et al. 2022 (arXiv:2203.11171), self-consistency dramatically improves CoT accuracy on tasks where multiple valid reasoning paths exist.

**Cost:** N model calls (typically N=5-20). Per-query cost N× single-call CoT.

**Accuracy lift:** +10-25% over single-chain CoT on math/logic benchmarks. The lift comes from filtering out reasoning errors via majority voting — if 80% of chains arrive at the same answer, it's much more likely correct than any single chain.

**Best for:** High-stakes individual queries where accuracy matters more than per-query cost. Math competition problems, scientific reasoning, code generation for critical functions. Cost makes it impractical for high-volume workloads.

**Mechanic:** Model explores multiple reasoning branches, evaluates intermediate states, backtracks from dead ends. Per Yao et al. 2023 (arXiv:2305.10601), this generalizes single-chain CoT to a search problem over reasoning trees.

**Cost:** 10-100× single-shot. Multiple model calls for branch generation, branch evaluation, and final answer synthesis.

**Accuracy lift:** +30-70% over single-chain CoT on hard reasoning tasks (Game of 24, creative writing with constraints, crossword puzzles). The lift is substantial but the cost premium is dramatic.

**Best for:** Genuinely hard reasoning tasks where single-chain CoT fails consistently and the per-query value is high enough to justify the cost. Not for high-volume workloads.

**Mechanic:** Before solving the specific problem, the model first generates a more abstract framing (the underlying principle, the general approach, the relevant domain knowledge). Then solves the specific problem with that abstraction loaded as context. Per Zheng et al. 2024 (arXiv:2310.06117), this 'step back' improves accuracy on tasks requiring domain knowledge recall.

**Cost:** 2 model calls (abstraction generation + specific problem solving). Per-query cost 2-3× single-shot.

**Accuracy lift:** +10-25% on knowledge-intensive reasoning tasks (physics problems requiring concept identification, multi-step inference requiring domain knowledge). Smaller lift on pure computation tasks.

**Best for:** Tasks where the model has the underlying knowledge but struggles to access it from a specific problem statement. The step-back generates the bridge to the relevant knowledge.

Picking CoT variant by familiarity: team uses whichever they first read about. Often zero-shot CoT for problems that would benefit from few-shot, or Tree of Thoughts on workloads where zero-shot CoT would suffice at 1/100th the cost.
Picking CoT variant by workload + stakes: zero-shot for general reasoning. Few-shot for domain-specific reasoning patterns. Self-consistency for high-stakes individual queries. ToT for genuinely hard problems with high per-query value. Step-back for knowledge-intensive reasoning. 5-100× cost differences map to real workload-specific value.