How to Use Tree-of-Thought Prompting

Tree-of-thought treats problem solving as a search. Instead of producing one linear chain of reasoning and committing to it, the model generates several candidate "thoughts" at each step, evaluates how promising each is, and only continues down the branches worth pursuing. Weak branches are pruned; strong ones are expanded. The result is a tree, and the answer is the best leaf.

The key difference from plain chain-of-thought is the explicit evaluation and backtracking. CoT is one path; if it goes wrong early, the whole answer is wrong. ToT keeps multiple paths alive and can abandon a dead end, which is why Yao et al. 2023 showed large gains on tasks where lookahead and backtracking matter, like the Game of 24 and creative planning.

In a research setting ToT runs as an actual search algorithm (breadth-first or depth-first) with a separate evaluator model. In day-to-day prompting you usually approximate it inside a single conversation: ask for branches, ask for scores, then ask for expansion of the winner. That captures most of the benefit without orchestration code.

ToT earns its cost on problems where the first idea is frequently the wrong idea and exploring alternatives pays off:

**Planning and strategy.** Choosing a launch sequence, an architecture, or an itinerary where several options must be compared against constraints before committing.

**Puzzles and constraint satisfaction.** Anything requiring lookahead — scheduling, the Game of 24, logic grids — where a greedy first move can paint you into a corner.

**Creative generation with selection.** Brainstorm several headlines, story openings, or product concepts, then rate and develop the best, instead of accepting the first.

Skip ToT for simple, single-path tasks — a lookup, a one-step calculation, or a well-specified rewrite. There, the branching and scoring just burn tokens and latency. And on a frontier reasoning model (GPT-5.5 thinking mode or Claude extended thinking), much of this exploration already happens internally, so a clear problem statement often beats a hand-rolled tree. Reach for explicit ToT on faster, non-reasoning tiers, or when you want the alternatives and scores visible for auditing.

Use tree-of-thought when: the problem benefits from exploring and comparing multiple paths before committing — planning, puzzles, constraint problems, or brainstorm-then-select tasks — and you want the alternatives visible.
Skip it when: the task is single-path or trivial, latency and token budget are tight, or you're on a frontier reasoning model that already explores internally — a clear problem statement usually wins there.

Here is a direct prompt that tends to grab the first plausible answer:

``` We have $5,000 and 6 weeks to get 100 paying signups for a B2B SaaS tool. What's the plan? ```

The model usually returns one generic plan ("run some ads, post on LinkedIn") with no comparison of alternatives. Now the tree-of-thought version:

``` Goal: 100 paying B2B SaaS signups in 6 weeks on a $5,000 budget. Step 1 — Branch: Propose 3 distinct go-to-market strategies (A, B, C). Each must use a different primary channel and a different core tactic. Step 2 — Evaluate: For each branch, rate 1-10 on (a) fit to a 6-week timeline, (b) fit to a $5,000 budget, (c) probability of hitting 100 signups. Give a one-line justification per score and a total. Step 3 — Prune & expand: Pick the highest-total branch. If two are close, briefly say why you chose one. Then develop it into a week-by-week plan with owners, budget split, and a single success metric per week. ```

The three stages map exactly to the ToT loop in Yao et al. 2023: generate thoughts, evaluate them, expand the best. The visible scores let you audit the choice — and override it if you disagree with a rating.

A lighter variant for creative work: "Write 4 different opening lines for this landing page. Score each for clarity and curiosity (1-5). Then rewrite the top scorer three different ways." Same branch-evaluate-expand shape, smaller tree.

These three techniques sit on a spectrum of how much they explore. Chain-of-thought (Wei et al. 2022) is a single reasoning path. Self-consistency (arXiv:2203.11171) samples many independent chains and takes the majority answer — exploration without structure. Tree-of-thought adds structure: it evaluates partial paths and decides which to extend, so compute is spent where it is most promising.

For most everyday tasks, CoT is enough. Move to self-consistency when you want a robustness vote on a question with one correct answer, and to ToT when the problem rewards deliberate search and backtracking. The DAIR.ai Prompt Engineering Guide has worked examples of all three if you want to go deeper.