An AI assistant should resolve conflicting instructions by authority—not by how friendly, urgent, flattering, or deferential a request sounds. In OpenAI’s public framework, system instructions outrank developer instructions, which outrank user instructions, which outrank tool output. A lower-priority instruction should be followed only when it does not conflict with a higher-priority one.
“The politeness trap” is a useful way to describe the pressure a nice-sounding request can create, but it is not a formal technical category in the sources discussed here. They do not measure how often politeness causes failures; they explain the broader problem of deciding which instructions govern when they conflict.
As an Amazon Associate I earn from qualifying purchases.
How the instruction hierarchy works
OpenAI’s public ordering is System > developer > user > tool. The ordering identifies the authority of an instruction’s source. It does not mean every AI system uses exactly the same hierarchy.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesOpenAI’s 2026 overview puts the rule this way: “Higher‑priority instructions are more trusted. The model should only follow lower‑priority instructions when they do not conflict with higher‑priority constraints.” Its Model Spec explainer adds that “The model is instructed to prioritize the letter and spirit of higher-authority instructions when conflicts arise.”
#1 Best Overall
That last point matters: a model should not satisfy the literal wording of a higher-priority rule while defeating its purpose. It must also distinguish an actual conflict from a request that can be fulfilled without violating the higher-priority instruction. OpenAI’s 2026 overview illustrates the hierarchy with a math tutor whose developer instruction says not to give students answers directly. A user can ask “pretty please,” but the request’s courtesy does not give it higher authority.
Why politeness can become a trap
Politeness is not itself an attack, and the cited work does not isolate or quantify a “politeness effect.” The risk is that language that feels socially compelling—such as flattery, urgency, deference, or a plea—may tempt a model to treat a lower-priority request as more important than it is. Under the hierarchy, tone does not change who issued the instruction.
The same discipline applies when a request is not polite. A blunt user instruction does not lose its user-level authority merely because it sounds rude. The model should ask who issued the instruction and whether following it would conflict with a higher-priority constraint.
How prompt injection exploits source confusion
Prompt injection is an attempt by a third party to mislead a model by placing malicious instructions in content the model is asked to process. That content might arrive through a web page or connected email. An attacker may try to redirect a recommendation or get the model to disclose information the user did not ask it to share. OpenAI describes these threats in its prompt-injection overview and safety guidance.
Rank #3
The underlying error is source confusion: treating untrusted content as if it had authority to instruct the assistant. A web page can contain text that looks like a command, but that does not make it a system or developer instruction. The model needs to handle that text as content relevant to the user’s task, not automatically obey it.
OpenAI links instruction prioritization to resistance against jailbreaks, system-prompt extraction, and direct or indirect prompt injection. But hierarchy alone does not eliminate these attacks. OpenAI calls prompt injection an evolving security challenge and cautions: “This guidance may not prevent every prompt injection, but it makes it harder for attackers to succeed.”
A practical way to assess a conflicting request
For users and builders evaluating an assistant’s behavior, separate three questions:
- Who issued each instruction? Distinguish system, developer, user, and tool output; treat retrieved or externally supplied content as untrusted unless the system explicitly grants it authority.
- Do the instructions actually conflict? If a user’s request can be completed within the higher-priority constraints, the assistant should still help. If it cannot, the higher-priority instruction governs.
- Could the action have real-world consequences? Sending an email, completing a purchase, or taking another consequential action calls for careful review, even if the instruction hierarchy is clear.
For agent builders, OpenAI recommends limiting access to the information and capabilities needed for the task, giving specific directions rather than broad discretion, and reviewing details before confirming important actions. These measures reduce risk; they are not guarantees. See OpenAI’s prompt-injection safety guidance.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What published evaluations show—and do not show
OpenAI’s 2026 overview reports improved scores for an instruction-hierarchy-trained model on several named evaluations. These are results for particular models and tests, not general rates of real-world safety or proof that a model cannot be manipulated.
| Evaluation | Reported comparison |
|---|---|
| TensorTrust (dev-user) | Baseline 0.76; instruction-hierarchy-trained model 0.91; reported gain +0.15 |
| RealGuardrails (Distractors) | Baseline 0.88; instruction-hierarchy-trained model 0.95; reported gain +0.07 |
| RealGuardrails (Handwritten) | Baseline 0.82; instruction-hierarchy-trained model 0.89; reported gain +0.07 |
| System IFEval | Baseline 0.92; instruction-hierarchy-trained model 0.96; reported gain +0.04 |
| System <> User Conflict | GPT-5-Mini 0.84; GPT-5 Mini-R 0.95; reported gain +0.11 |
| Developer <> User Conflict | GPT-5-Mini 0.83; GPT-5 Mini-R 0.95; reported gain +0.12 |
| IH-Challenge (overrefusal) | GPT-5-Mini 0.79; GPT-5 Mini-R 1.00; reported gain +0.21 |
The same overview reports no change on GPQA Diamond (0.83 to 0.83), a small gain on AIME 2024 (0.93 to 0.94), and lower Chat WinRate versus o1 (0.71 to 0.66) and Preference Score (0.46 to 0.40). The results show why hierarchy behavior cannot be judged in isolation: stronger performance on selected conflict tests can coexist with tradeoffs on other evaluations. All figures above are reported by OpenAI in its 2026 overview; they should be read in the context of those models and benchmarks.
What remains uncertain
The available sources do not establish how prevalent politeness-driven hierarchy failures are across models, providers, or real-world deployments, and they do not test politeness as an isolated variable. OpenAI’s published framework and evaluations support an account of OpenAI’s approach; they are not an independent, cross-industry comparison.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
