How does prompt injection attack affect ChatGPT architecture and how is it mitigated?

Prompt injection manipulates model behavior by overriding system instructions through malicious input, mitigated using isolation and filtering layers.

Prompt injection occurs when a user crafts input that manipulates the model into ignoring system-level instructions. Since LLMs treat all text as input tokens, there is no strict separation between system and user instructions internally.

Architectural defenses include instruction hierarchy enforcement, system prompt isolation, input sanitization, retrieval filtering, and tool-use restrictions. More advanced systems use policy models that evaluate whether a prompt is safe before passing it to the main LLM.

This is a core security challenge in LLM-based applications.