Chain-of-Thought Prompting: A Practical Guide (2026)

At its simplest, chain-of-thought (CoT) means asking the model to show its work. Instead of "What's the answer?", you ask "Reason through this step by step, then give the answer." The model generates a sequence of intermediate steps, and the final answer falls out of that sequence.

There are two common forms. Zero-shot CoT adds a trigger phrase like "Let's think step by step" with no examples. Few-shot CoT shows the model one or more worked examples that include the reasoning, then asks it to solve a new problem the same way. The original Wei et al. 2022 paper demonstrated the few-shot form; the zero-shot trigger came shortly after and is now the everyday default.

The mechanism is straightforward: generating intermediate tokens gives the model more computation to allocate to the problem and forces each conclusion to be conditioned on an explicit prior step. Errors that slip through a single-shot answer often surface — and get corrected — when the reasoning is spelled out.

CoT earns its keep on problems with multiple dependent steps. The clearest wins, consistent with the findings in Wei et al. 2022:

**Arithmetic and word problems.** Anything where the answer depends on a chain of calculations. Showing the steps catches the off-by-one and order-of-operations mistakes that one-shot answers make.

**Multi-hop reasoning.** Questions that require combining several facts ("X happened before Y, and Y caused Z, so..."). Laying out the hops keeps the model from skipping a link.

**Logical deduction and constraint problems.** Puzzles, scheduling, eligibility rules — anywhere the answer must satisfy several conditions at once.

**Decisions with explicit criteria.** When you want the model to weigh options against stated criteria rather than pattern-match to a plausible-sounding answer, asking it to evaluate each criterion in turn improves consistency.

CoT is not free, and it is not always useful. Skip it — or expect little benefit — in these cases:

**Simple lookups and single-step tasks.** Asking a model to "reason step by step" about a one-step fact just burns tokens and latency for no accuracy gain.

**Open-ended creative work.** Forcing explicit reasoning can flatten tone and make output feel mechanical. Creative tasks usually want strong instructions, not a visible deduction chain.

**Modern reasoning models already do it internally.** This is the biggest 2026 shift. Today's frontier reasoning models — including OpenAI's reasoning-oriented models and Anthropic's Claude Opus 4.8 and Sonnet 4.6 — perform extensive internal step-by-step reasoning before they answer. Bolting an explicit "think step by step" instruction onto these models often adds little, because the work is already happening under the hood. See the Claude prompt engineering overview and OpenAI's prompting guide for current model-specific guidance.

The practical rule for 2026: reach for explicit CoT when you are on a fast, non-reasoning model (the cheaper, lower-latency tiers) and the task has multiple steps. On a dedicated reasoning model, lead with a clear problem statement and let the model reason; add explicit step instructions only if the output is still skipping steps.

Use explicit CoT when: the task has multiple dependent steps (math, logic, multi-hop), AND you're on a fast non-reasoning model tier where the reasoning isn't already happening internally. Also useful when you need the reasoning visible for auditing or debugging.
Skip explicit CoT when: the task is a single-step lookup, the output is creative/stylistic, or you're already on a frontier reasoning model that does step-by-step reasoning internally — in which case a clear problem statement usually beats a bolted-on 'think step by step'.

Here is a one-step-answer prompt that frequently trips up fast, non-reasoning models on multi-step math:

``` A store sells notebooks at $3 each. If you buy 10 or more, you get 15% off the whole order. I'm buying 12. What's the total? ```

Direct answers sometimes apply the discount to only part of the order or miscompute the percentage. Now the chain-of-thought version:

``` A store sells notebooks at $3 each. Buying 10 or more gives 15% off the whole order. I'm buying 12. Reason step by step: 1. Compute the pre-discount subtotal. 2. Check whether the 10+ discount applies. 3. Apply the discount to the full subtotal. 4. State the final total. Then give the final total on its own line as: TOTAL: $X.XX ```

The numbered scaffold forces each step and pins the output format so the answer is easy to extract. Expected reasoning:

> 1. Subtotal = 12 x $3 = $36.00 > 2. 12 >= 10, so the 15% discount applies. > 3. Discount = $36.00 x 0.15 = $5.40; $36.00 - $5.40 = $30.60 > 4. TOTAL: $30.60

---

A second pattern: few-shot CoT for a classification-with-reasoning task. Show one worked example, then ask for the same structure:

``` Classify each support ticket as URGENT, NORMAL, or LOW. Explain your reasoning in one sentence, then give the label. Ticket: "Checkout has been throwing a 500 error for all users for 20 minutes." Reasoning: A total checkout outage blocks revenue for every customer, so it is the highest severity. Label: URGENT Ticket: "Can you change the font on my invoice PDF?" Reasoning: Label: ```

The worked example teaches the reasoning style and output shape at once — far more reliable than describing the format in prose.

Chain-of-thought is the foundation for several more advanced patterns. Self-consistency samples multiple reasoning chains and takes the majority answer. Tree of Thoughts (arXiv:2305.10601) explores a branching tree of reasoning paths and prunes weak branches. ReAct (arXiv:2210.03629) interleaves reasoning steps with tool calls, which is the backbone of most modern agents.

For everyday work you rarely need the exotic variants. Plain zero-shot CoT ("reason step by step, then answer") on a multi-step task covers the large majority of real use. Add a worked example (few-shot CoT) when you also need to pin a specific reasoning style or output format.