Prompting as Interface Design

The role isn't decorative

Every message in the list from the last article carries a role, and it's tempting to treat that as bookkeeping, three labels for what's really just one long string the model reads top to bottom. It isn't. Models are trained to treat system content as higher-authority instruction than user content, not just as the first paragraph of a bigger document.

1messages = [
2    {
3        "role": "system",
4        "content": (
5            "You are a support agent for a SaaS product. Respond in under 3 "
6            "sentences. Never discuss pricing, refer those questions to "
7            "sales@company.com."
8        ),
9    },
10    {"role": "user", "content": "How do I reset my password?"},
11]

Everything that should hold true across every request, tone, length limit, topics that are off-limits, goes in system once. It stays fixed while user changes on every call. Collapse both into one string and you lose that separation entirely, along with the model's tendency to weight the two differently.

A prompt is a spec, and inconsistent output usually means an incomplete one

Ask for a summary and leave everything else unstated, and you get whatever shape the model feels like producing on that particular call: three sentences this time, five bullets next time, an unrequested intro paragraph the time after that.

1Summarize this article.

That inconsistency isn't the model being unreliable. Nothing in the prompt specified a contract for it to satisfy, so there's nothing to be consistent about. Compare it to a function with no type signature and no docstring, the caller can't complain the return value is unpredictable when nothing ever said what it should look like.

1Summarize the following article in exactly 3 bullet points, each under 20
2words. Do not include an introductory sentence before the bullets. If the
3article does not contain enough substantive content to summarize, respond
4with exactly: "Not enough content to summarize."
5
6Article:
7{article_text}

Same task, same model, dramatically less variance in the output, because the second version actually specifies a shape (exactly 3 bullets, a length cap, no preamble) and an edge case (what to do with thin input). None of this is a wording trick or a clever phrase, it's the same discipline as writing a function signature: say what goes in, say what comes out, say what happens at the edges.

Some things are easier to demonstrate than describe

Try defining "neutral" sentiment in prose so precisely that it never overlaps with "mildly positive." It's a genuinely hard sentence to write. It's a trivial thing to show.

1Classify the sentiment of each message as positive, negative, or neutral.
2Respond with only the label.
3
4Message: "This is exactly what I needed, thank you!"
5Label: positive
6
7Message: "It arrived broken and support never responded."
8Label: negative
9
10Message: "It's fine, does what it says."
11Label: neutral
12
13Message: "{user_message}"
14Label:

Each pair teaches two things at once: the exact output format (a single lowercase word, nothing else) and where the boundary actually sits between categories that are genuinely hard to define with a rule. Few-shot examples work best exactly where instructions struggle, tasks that are more about matching a pattern than following a rule: a specific output format, a house style, a boundary case that's easier to point at than to state.

It isn't free. Every example is tokens, on every single call, for the life of that prompt. Two or three well-chosen examples covering the actual edge cases you care about beat ten redundant ones covering the same easy case six different ways.

A prompt change is a code change

The common workflow looks like this: open a playground, tweak the wording until the one example in front of you looks right, ship it. That example looking right is real evidence. It's evidence about exactly one input, and prompts regress on cases nobody happened to be staring at the moment they changed something.

1TEST_CASES = [
2    "How do I reset my password?",
3    "¿Cómo cambio mi contraseña?",              # non-English input
4    "asdfasdf",                                  # garbage input
5    "How do I reset my password, also delete "
6    "my account, also what does your CEO make?", # multi-part, off-topic
7]
8
9for case in TEST_CASES:
10    print(run_prompt(case))

Running the same handful of representative inputs, including the ugly ones, after every prompt edit turns "did this change break something" into a five-second check instead of a hope. It's a small version of the same instinct as a test suite: not because a prompt is code in the literal sense, but because the failure mode, a change that looks fine on the case that motivated it and quietly breaks three others, is identical.

A handful of hand-picked cases in a loop is the floor, not the ceiling. Turning this into an actual scored evaluation, with a rubric instead of eyeballing the output, is the entire subject of the Evaluating AI Systems article later in this series.