What is prompt injection?
Prompt injection is an attack in which malicious instructions influence an AI model's response or intended behavior. It may be direct—for example, a user tries to override an assistant's rules—or indirect, when instructions are hidden inside a webpage, email, document, search result, or other content that the assistant reads.
This matters most when an AI system can access private data or take actions through tools. A summarization bot may only return an unsafe answer; an agent connected to email, files, internal systems, or payment workflows may have a much larger impact if it follows hostile instructions.
OWASP identifies prompt injection as a key risk in its GenAI security guidance.
Why a system prompt is not a security boundary
A developer can instruct a model to treat external text as untrusted, refuse certain requests, or ask before taking action. These are useful safeguards, but they are not equivalent to an enforceable permission system. The model processes instructions and data as part of the same context and can still be influenced by malicious content.
Do not depend on a phrase such as “ignore all malicious instructions” to protect a database, secret, or administrative action. Enforce security outside the model with ordinary access controls, data minimization, validation, and explicit authorization.
Common scenarios
An AI assistant that reads webpages
A webpage may contain instructions aimed at the AI rather than the human reader. If the assistant follows those instructions, it could return misleading information or attempt to use its connected tools in an unintended way.
An email or document summarizer
An attacker may send a message containing hostile instructions designed to alter a summary or influence what the assistant does next. Treat the email or document as data, not as a trusted source of application policy.
A support agent connected to internal systems
An agent that can search customer records, issue refunds, or change account settings creates a higher-impact risk. If a model is manipulated, excessive permissions can turn a misleading response into an unauthorized action.
How to reduce prompt-injection risk
1. Apply least privilege to every tool
Give an agent only the tools and data it needs for its task. A summarizer rarely needs permission to send email, delete records, or execute arbitrary code. Separate read-only tools from write-capable tools and use scoped credentials.
2. Enforce authorization in the tool layer
Every API call should independently verify the user's identity, permissions, record scope, and intended action. Do not assume that because the model decided to call a tool, the call is authorized. Validate arguments and reject operations outside the permitted schema.
3. Require human approval for consequential actions
Use explicit approval before external messages are sent, data is deleted or exported, money moves, privileges change, or production systems are modified. Display the exact proposed action and its target to the reviewer; do not ask for a vague approval of a broad plan.
4. Separate instructions from retrieved content
Label external documents, user uploads, and retrieved webpage content as untrusted. Keep system policy separate in the application's design, limit the amount of content placed into context, and avoid treating text from external sources as authority to invoke tools.
This can reduce risk, but prompt formatting alone is not a complete defense.
5. Minimize sensitive context
Do not pass passwords, API keys, full customer databases, or unrelated personal data to a model when the task does not require them. Retrieve the smallest relevant set of records and redact sensitive fields before model processing.
6. Validate model output before using it
Treat model-produced code, SQL, HTML, URLs, filenames, and tool arguments as untrusted input. Apply strict schemas and allowlists, use parameterized queries, encode HTML correctly, and never execute generated commands without an isolated, carefully controlled design.
7. Log, test, and monitor
Record tool calls, authorization decisions, approval outcomes, and relevant failures while avoiding secrets in logs. Build adversarial tests with hostile instructions in user messages, retrieved webpages, and uploaded files. Monitor unexpected tool use, unusual data access, repeated refusals, and high-cost activity.
A simple secure-agent architecture
A safer workflow separates the model from the systems it can affect:
1. The model proposes an action with structured arguments. 2. Application code validates the arguments against a strict schema. 3. The authorization layer checks identity, permissions, and scope. 4. A human approves high-impact actions when required. 5. A narrowly privileged tool executes the operation. 6. The application records the outcome and returns only necessary information.
The model can help decide what to propose, but deterministic application code should enforce what is allowed.
Does retrieval-augmented generation (RAG) prevent prompt injection?
No. RAG can improve relevance by retrieving supporting documents, but retrieved documents remain untrusted content and may contain malicious instructions. Access controls on the retrieval layer, source validation, context minimization, safe tool design, and output checks are still needed.
A practical pre-launch checklist
- Inventory every tool and the permissions it grants.
- Confirm each tool enforces authorization independently.
- Remove unnecessary write access and secrets from model context.
- Require approval for high-impact or externally visible actions.
- Validate inputs and outputs with strict schemas and allowlists.
- Test direct and indirect prompt injection in a safe environment.
- Monitor agent activity and prepare a rollback or disable switch.
For a deeper framework, use the OWASP GenAI Top 10 for 2026 and the OWASP prompt injection prevention cheat sheet.
Final takeaway
Prompt injection is not solved by writing a stronger system prompt. Secure AI applications by limiting what an agent can access, enforcing permissions outside the model, validating every tool call, requiring approval for consequential actions, and continuously testing hostile inputs. Design the system so that even a manipulated model cannot exceed its authority.