How to Write Prompts for Classification

Classification is the task of assigning each input to one (or more) labels from a fixed set — routing a support ticket to a queue, tagging feedback sentiment, flagging a document type. The failure modes are: the model invents a label outside your set, picks inconsistently on borderline cases, or returns a paragraph of justification when you wanted a single token.

Three design choices fix nearly all of it. First, a **closed label set** — the model may only return values from your list, never freeform. Second, **clear label definitions** so 'urgent' and 'high priority' aren't interpreted interchangeably. Third, **examples that teach the boundaries**, especially the cases where two labels compete. Most people add easy examples; the value is in the hard ones.

The few-shot approach — showing labeled examples in the prompt so the model generalizes the pattern — is the foundational technique from Brown et al. 2020. The provider prompt guides (OpenAI, Anthropic) and the DAIR.ai Prompt Engineering Guide all treat few-shot labeling as the default for classification.

Keep the label set small, distinct, and mutually exclusive where possible. Overlapping labels ('bug' vs. 'technical issue') force the model to guess and make your results inconsistent. If two labels keep getting confused, that's a signal to merge them or sharpen their definitions.

Define every label in one line, stated in terms of an observable test the model can apply: "billing — the message is about charges, invoices, refunds, or payment methods." Definitions do more work than the label names themselves, because a bare word like 'general' means nothing without a boundary.

Always include an escape-hatch label — 'other', 'none', or 'unsure' — and define exactly when to use it: "use 'other' only if the message fits none of the labels above." Without it, the model crams genuinely-unclassifiable inputs into the nearest label, which is worse than an honest 'other' you can review.

Decide single-label vs. multi-label up front and state it. If an input can carry several tags, ask for an array and say so; if it must be exactly one, say 'return exactly one label.' This is the same enum discipline used in data extraction — a closed set with a defined fallback.

For most classifiers, a handful of examples per label is plenty — accuracy gains flatten quickly after that, and more examples cost tokens and latency. What matters is which examples you pick, not how many.

Choose examples that sit on the **decision boundaries**: the sarcastic-but-positive review, the feature request disguised as a complaint, the ticket that mentions billing but is really about a bug. These are the cases the model gets wrong without guidance, and one good boundary example teaches more than ten obvious ones. Balance the set so each label appears a similar number of times — a lopsided example set biases the model toward the over-represented label.

Format examples identically to the output you want: input, then label, nothing else. Mixing formats in your examples teaches the model to vary its output. If your labels are subtle, a short reason before the label can help on a fast non-reasoning model — but on a frontier reasoning model that already reasons internally (see chain-of-thought prompting), a bare label is usually enough and far cheaper to parse.

When the label set or examples are stable across a high-volume job, cache the static portion of the prompt to cut cost — see LLM caching strategies.

Edge cases are where naive classifiers fall apart. Plan for four of them explicitly in the prompt. **Ambiguous inputs** that could be two labels — tell the model how to break ties ('if both billing and technical apply, choose the primary intent'). **Out-of-scope inputs** — route to your 'other' label. **Empty or garbage input** — define a label or null for it rather than forcing a guess. **Multi-intent inputs** — decide whether to allow multiple labels or pick the dominant one.

Add a confidence signal when misclassification is costly. Asking for a confidence level (high/medium/low) alongside the label lets you auto-accept high-confidence results and route the rest to human review — a far better pipeline than trusting every label blindly.

Validate in code against the closed set: any label not in your list is an automatic reject-and-retry. This catches the rare invented label before it pollutes downstream data, the same validate-and-retry loop recommended for structured output.

For sensitive classification — triaging medical messages, legal documents, or financial complaints — this is informational only and not professional advice. Never input personal, health, or confidential client data into a public chatbot, and have a qualified person review consequential classifications.

Here is a vague prompt that returns inconsistent, unparseable labels:

``` What category is this support message? {message} ```

It invents categories, explains itself at length, and labels borderline cases differently each run. Now the closed-set, few-shot version with an escape hatch:

``` Classify the support message into EXACTLY ONE label from this list. Return only the label — no explanation. Labels: - billing — charges, invoices, refunds, payment methods - technical — bugs, errors, things not working - account — login, password, profile, access - other — fits none of the above If two labels could apply, choose the primary intent. Use "other" only if none fit. Examples: Message: "I was charged twice this month." Label: billing Message: "The export button does nothing and I paid for this!" Label: technical Message: "Can't log in after the update." Label: account Message: {message} Label: ```

Notice the second example: it mentions payment but the primary intent is a bug, so it's labeled technical. That single boundary example does more for accuracy than any amount of instruction text. Validate the returned label against the four-item list in code and retry on anything else.