Prompt engineering that actually works

Prompt engineering has a reputation problem. Search for it and you find lists of secret phrases, role-play tricks and threats in capital letters. Most of that does not survive contact with a new model version. What does survive is plainer: say clearly what you want, give the model the context it lacks, show examples carefully, structure long inputs, and check your changes against tests. This article is about that durable core.

Write a brief, not a wish

A language model sees only the text you send it. It does not know your product, your users, your coding conventions or why this request exists. A useful test: if you handed the prompt to a smart contractor on their first day, could they do the job well?

Be explicit about:

• The task — what exactly should be produced, and what counts as done.
• The audience — a summary for a lawyer differs from one for a customer.
• Constraints — length, tone, what to avoid, what to do when information is missing.
• The output format — prose, bullet list, a JSON object with named fields.

Give the “why”

Instructions with reasons generalise better than bare rules. “Do not use abbreviations” leaves the model guessing at edge cases. “Readers are non-native English speakers, so avoid abbreviations and idioms” also steers word choice you never mentioned. The reason is extra context the model can reason from.

Say what to do, not only what not to do

“Don’t be verbose” tells the model what to avoid but not what to aim for. “Answer in two or three sentences, then stop” is easier to follow and easier to check.

Examples: powerful and easily over-learned

Including a few worked input-output pairs — few-shot prompting — is one of the most effective techniques available. The GPT-3 paper, Language Models are Few-Shot Learners (Brown et al., 2020), showed large models could pick up tasks from examples in the prompt alone.

The catch is that models imitate examples very faithfully, including features you did not intend to teach:

• If every example is short, outputs get short.
• If most example labels are “positive”, predictions skew positive.
• If examples share a topic or phrasing, outputs drift toward it.

So choose examples that are diverse (different lengths, topics and difficulty), representative of real inputs, and that include edge cases such as ambiguous or empty inputs. Clearly mark them as examples so the model does not treat their content as part of the current request.

Structure long inputs

When a prompt mixes instructions, reference documents, examples and user data, the model can confuse which is which. XML-style tags draw clear boundaries:

<instructions>
Answer using only the contract below. If the answer is not in it, say so.
</instructions>

<contract>
...full contract text...
</contract>

<question>
What is the notice period for termination?
</question>

Tags have no special meaning to most models beyond being clear delimiters, but that clarity matters. You can refer to sections by name (“using the contract above”), ask for output inside tags, and parse those sections out in code. Delimiting untrusted content also helps a little against it being read as instructions — though it is not a security boundary against prompt injection.

With very long documents, a widely recommended pattern is to put the documents first and the question near the end. As with every tip here, confirm it on your own task.

System prompts

Most chat APIs separate a system prompt from the conversation turns. Use it for stable, app-wide context: the role the model plays, who the users are, tone, rules and formatting conventions. Put per-request material — the user’s question, retrieved documents — in the user turn. This keeps the standing instructions reusable and makes it easier to see what changed between requests.

Let the model think before it answers

Models generate text one token at a time, and each token is conditioned on what came before. For multi-step problems, asking the model to work through the problem before committing to an answer often improves accuracy. Wei et al. (2022) named this chain-of-thought prompting and showed large gains on arithmetic and reasoning benchmarks.

Two practical points:

1. Order matters. Reasoning must come before the final answer. An answer followed by an explanation is a justification, not reasoning.
2. Separate the parts. Ask for reasoning in one tagged section and the answer in another, so your code can extract just the answer.

Many current models have built-in “thinking” or reasoning modes that do this for you. Either way, reasoning costs tokens, latency and money, and the written reasoning is not guaranteed to faithfully reflect how the model reached its answer. Use it where it measurably helps.

Break big jobs into steps

One enormous prompt that extracts, analyses, critiques and formats is hard to debug. Prompt chaining splits it into stages whose outputs feed the next. Each stage is simpler, can use a different model or temperature, and can be tested on its own.

Prompts are code

The most important habit is not a phrasing trick. It is process.

A prompt change is a code change with non-local effects. Fixing one example can quietly break ten others.

Treat prompts accordingly:

• Keep them in version control, not pasted into a dashboard nobody reviews.
• Maintain a test set of real inputs, including the ones that failed before.
• Change one thing at a time and rerun the whole set.
• Write checks where possible: does the output parse, include required fields, stay under a length, cite a source?
• Read outputs yourself; metrics miss failures you have not thought of yet.

Anthropic’s own prompt engineering documentation puts this first: define success criteria and a way to test against them before you start tuning wording. Without tests you cannot tell a real improvement from a lucky sample.

Honest limits

Prompting cannot add knowledge the model lacks (that is what retrieval is for), cannot guarantee a format (validate in code), and cannot make a model reliably do a task beyond its ability. Sometimes the right fix is a different model, a smaller task, or a system around the model. Good prompt engineering is knowing when you have hit those limits — and having the tests to show it.