Stage 3: Instruction/Input Weaponization
Objective
Up to this point, the attacker is shaping behavior. Stage 3 is when the payload arrives, and that payload is not code. Stage 3 is the conversion of attacker intent into inputs that the AI system will treat as authoritative instructions.
This is the AI-native equivalent of:
Weaponization + Delivery (classical kill chain)
Initial Access (MITRE ATT&CK)
Why Instruction/Input Weaponization is Different from Classic Exploits
Payload = binary
Payload = language
Exploit targets memory
Exploit targets reasoning
Delivery via network/file
Delivery via prompt/data
Detection via signatures
Detection via semantics
No crash, no exception, and no exploit primitive. The system works as designed, but toward the wrong goal.
Core Techniques: Instruction/Input Weaponization
Stage 3 to Stage 4 transformation
Stage 3 ends when:
The malicious instruction is accepted as valid context
The system no longer questions its legitimacy
Stage 4 begins when:
The model reasons over that instruction
Planning and tool selection start
Stage 3 is delivery and Stage 4 is detonation.
What Stage 3 Looks Like in Logs
“Helpful” explanations before actions. The model restates or justifies steps that were never requested, which signals injected intent.
Long contextual documents preceding tool calls. The model uses large retrieved text blocks that may contain hidden instructions.
Sudden goal changes without explicit user request. The model shifts direction based on context it should not have treated as commands.
Instructions embedded in retrieved text. The model executes behavior influenced by data sources instead of the actual user prompt.
Most Abused Stage in Real Incidents
Stage 3 is the most abused stage in real incidents because it requires no bypass. The attacker leverages normal model behavior without defeating safeguards.
It requires no vulnerability. The model treats malicious inputs as valid context and follows them.
It requires no permissions. Attackers only need the ability to supply text that the model will later consume.
It works as designed. The model is doing exactly what its training optimizes it to do.
It scales across users. Persistent or shared inputs affect many sessions, amplifying impact.
If attackers control inputs, they control outcomes. The model follows whatever it interprets as the intended task.
Defensive approaches
Instruction/data separation
Systems must isolate commands from content so the model cannot treat data as instructions.
External policy enforcement
Guardrails outside the model ensure constraints remain intact even if the model misinterprets context.
Tool gating outside the model
Decisions to call tools must be validated externally rather than generated solely by the model.
Memory write controls.
Stored information must be filtered to prevent accidental retention of instructions.
Retrieval sanitization
Retrieved text must be cleaned so embedded commands do not reach the model as actionable context.
Last updated