UnifAI Policies
UnifAI policies are your built-in guardrails for AI security and compliance. Instead of manually tracking complex regulations, UnifAI automates policy enforcement across your AI ecosystem, so you can innovate without risk.
Every policy is mapped to global standards like OWASP and EU AI Act, ensuring your AI systems stay compliant and resilient.
Lineaje Out-of-the-Box Policies
Lineaje provides out-of-the-box policies across five categories:
AI Threats and Exploits
Data Security and Privacy
Identity and Access Control
Vulnerability
Skills
In UnifAI, enter the prompt View all policies to see your policies.
AI Threats and Exploits
Blocks prompt injection, adversarial inputs, and unsafe model behavior before they reach your AI apps.
Do not allow malicious content via hidden prompts
AI_APP_SEC_001
Violation Summary
Hidden or non-visible prompts detected in the system introduce risks of prompt injection, bypass of safety controls, and untraceable model behavior.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details
Hidden prompts introduce several risks including:
Undetectable prompt injection
Unpredictable, unsafe or incorrect output
Bypass safety and governance controls
Unsafe or inconsistent agent behavior
Regulatory and Ethical exposure
Attack Vector: Prompt
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
CERT-In AI Blueprint 2026
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Do not allow malicious content via encoded prompts
AI_APP_SEC_002
Violation Summary
Encoded prompts are instructions hidden inside obfuscated text, Base64, hex, zero-width characters, steganographic patterns, metadata, or structured payloads. They allow attackers or internal actors to bypass oversight, evade filters, or manipulate an AI system without detection.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details
Hidden prompts introduce several risks including:
Invisible prompt injection leading to unauthorized system behavior
Safety bypass (toxicity, policy evasion, jailbreaks)
Leakage of sensitive data or internal system instructions
Corruption of downstream workflows due to manipulated outputs
Violations of transparency, record-keeping, and explainability requirements
Attack Vector: Prompt
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
CERT-In AI Blueprint 2026
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Use only LLMs from the organization’s approved list
AI_APP_SEC_006
Violation Summary Using an LLM that is not on the organization’s approved list introduces uncontrolled security, privacy, compliance, and operational risks. Unapproved LLMs may have unknown data handling practices, insufficient security controls, unclear training or retention policies, weak contractual protections, or unvetted model behavior. This bypasses governance, procurement, and risk management processes, exposing the organization to data leakage, regulatory violations, vendor lock-in, and unpredictable AI behavior across agentic and automated workflows.
Affected Assets
LLM
AI Agent
Severity
High/Critical
Technical Details
Using an unapproved LLM introduces several risks including:
Uncontrolled processing, retention, or reuse of sensitive data and prompts
Unknown security posture, access controls, and logging practices
Potential training on proprietary or regulated data without consent
Incompatibility with organizational guardrails, monitoring, or audit tooling
Increased exposure to prompt injection, data leakage, or unsafe outputs
Breach of contractual, legal, or regulatory obligations
Loss of centralized governance, visibility, and incident response capability
Attack Vector: LLM selection / API usage outside approved platforms
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: High
Availability Impact: Medium (instability or service changes)
Framework
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
MCP server must validate and sanitize all input
AI_APP_SEC_014
Violation Summary
If an MCP server accepts input from clients, agents, or LLMs without validation or sanitization, it becomes vulnerable to malformed payloads, injection attacks, unauthorized tool invocation, unsafe command execution, and data corruption. Because MCP servers often expose high-privilege operations (file access, API calls, system actions), unvalidated input can be weaponized to manipulate workflows, escalate privileges, or deliver malicious instructions that compromise both the server environment and downstream systems.
Affected Assets
MCP Server
Severity
Critical
Input Validation
Validation ensures that input conforms to expected structure, type, length, format, and policy constraints before it is processed by the LLM. It answers the question if the input is allowed to be processed.
Examples of Validation
Rejecting prompts longer than a defined maximum length
Enforcing schema compliance (e.g., JSON with specific fields only)
Blocking inputs containing disallowed patterns (e.g., ignore previous instructions, system override)
Restricting input sources to authenticated or trusted origins
Ensuring prompts match an approved task or intent category
Input Sanitization
Sanitization transforms input to remove, neutralize, or normalize unsafe elements while preserving legitimate intent. Sanitization ensures that the input is made safe before processing.
Examples of Sanitization
Normalizing Unicode to remove obfuscation (e.g., leetspeak, homoglyphs)
Stripping zero-width or invisible characters
Decoding and inspecting encoded content (Base64, hex) before use
Escaping or isolating untrusted text so it cannot be interpreted as instructions
Removing or redacting sensitive data (PII, secrets)
Technical Details
Not validating or sanitizing MCP server input introduces several risks including:
Injection of malicious commands, payloads, or structured data into tools or system functions
Execution of unsafe or hallucinated instructions originating from LLM output
Unauthorized access or misuse of server-side capabilities and sensitive APIs
Corruption of data, resources, or operational workflows through malformed input
Increased attack surface for prompt-to-system escalation attacks
Loss of governance, auditability, and explainability of server-driven actions
Violations of integrity, safety, and regulatory obligations for high-risk functions
Attack Vector: MCP client → server input channel
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium to High (depending on server capabilities)
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
MCP clients must log all interactions with the MCP server
AI_APP_SEC_022
Violation Summary A missing or incomplete logging mechanism between the MCP client and MCP server creates a critical visibility and governance gap. MCP interactions often trigger high-privilege actions (tool execution, data access, workflow modification). Without proper logs, misuse, anomalies, attacks, or unauthorized system changes cannot be detected, investigated, or attributed. This results in opaque AI behavior, broken audit trails, and non-compliance with required traceability and transparency obligations.
Affected Assets
MCP Client
MCP Server
Severity
Critical
Technical Details
Failure to log MCP interactions introduces several risks including:
Undetectable misuse or abuse of MCP server tools
Inability to perform forensic investigation during an incident
Loss of accountability for AI-driven actions and decisions
Exposure to covert prompt injection or unauthorized system manipulation
Violations of traceability, transparency, and record-keeping requirements
Difficulty detecting anomalous behavior or lateral movement
Corruption of downstream workflows due to hidden actions
Attack Vector: MCP tool invocation / API interaction
Attack Complexity: Low
Privileges Required: None, when exploited via LLM-driven tool calls.
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
EU AI Act
NIST AI RMF
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Client must validate and sanitize any output from a MCP server
AI_APP_SEC_023
Violation Summary
When an MCP client consumes output from an MCP server without validation or sanitization, it exposes the AI system and downstream components to malformed data, malicious payloads, injection attacks, hallucinated instructions, and unsafe tool execution. MCP server output may include structured data, commands, untrusted text, or model-generated content. Without safeguards, unvalidated output can drive unsafe automated actions, corrupt workflows, or leak sensitive information.
Affected Assets
AI Agent
MCP Client
Severity
Critical
Input Validation
Validation ensures that input conforms to expected structure, type, length, format, and policy constraints before it is processed by the LLM. It answers the question if the input is allowed to be processed.
Examples of validation
Rejecting prompts longer than a defined maximum length
Enforcing schema compliance (e.g., JSON with specific fields only)
Blocking inputs containing disallowed patterns (e.g., ignore previous instructions, system override)
Restricting input sources to authenticated or trusted origins
Ensuring prompts match an approved task or intent category
Input Sanitization
Sanitization transforms input to remove, neutralize, or normalize unsafe elements while preserving legitimate intent. Sanitization ensures that the input is made safe before processing.
Examples of sanitization
Normalizing Unicode to remove obfuscation (e.g., leetspeak, homoglyphs)
Stripping zero-width or invisible characters
Decoding and inspecting encoded content (Base64, hex) before use
Escaping or isolating untrusted text so it cannot be interpreted as instructions
Removing or redacting sensitive data (PII, secrets)
Technical Details
Lack of output validation introduces several risks including:
Execution of harmful or unintended actions triggered by malformed MCP output
Injection of unsafe code, commands, or control sequences into downstream systems
Propagation of hallucinated, incorrect, or manipulated data
Leakage of sensitive information through unfiltered server responses
Corruption of business workflows or agent decision chains
Evasion of safety controls due to unmonitored tool responses
Violations of auditability, reliability, and compliance requirements
Attack Vector: MCP response / server-generated output
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium (via cascading workflow corruption)
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Do not use LLMs from the organization's disallowed list
AI_APP_SEC_028
Violation Summary Using an LLM that is explicitly on the organization’s block list represents a deliberate bypass of governance, security, and risk controls. Block-listed LLMs are typically prohibited due to known deficiencies such as unsafe data handling, unacceptable training or retention practices, lack of contractual protections, regulatory exposure, weak security posture, or demonstrated unsafe behavior. Their use introduces severe security, privacy, compliance, and reputational risks and undermines centralized AI governance.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details
Using a block-listed LLM introduces several risks including:
Known or previously identified data leakage, retention, or misuse risks
Exposure of sensitive, proprietary, or regulated data to untrusted providers
Circumvention of organizational security, legal, and compliance controls
Lack of auditability, logging, or incident response visibility
Increased likelihood of unsafe, biased, or non-compliant model behavior
Breach of regulatory, contractual, or internal policy obligations
Loss of trust in AI governance and enforcement mechanisms
Attack Vector: Unauthorized LLM selection / direct API or UI usage
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: High
Availability Impact: Medium
Framework
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Agent must validate, sanitize LLM output including for presence of eval or any dynamic code execution primitive in LLM output
AI_APP_SEC_029
Violation Summary
When LLM outputs are consumed without validation or sanitization, the system becomes vulnerable to unsafe instructions, hallucinated commands, malicious payloads, and untrusted code. This risk becomes critical when the LLM output may contain eval, shell commands, SQL statements, or other dynamic execution primitives. If such outputs pass directly into an interpreter, agent tool, or workflow engine, they can lead to arbitrary code execution, data exfiltration, workflow corruption, or full system compromise.
Affected Assets
AI Agent
Severity
Critical
Input Validation
Validation ensures that input conforms to expected structure, type, length, format, and policy constraints before it is processed by the LLM. It answers the question if the input is allowed to be processed.
Examples of Validation
Rejecting prompts longer than a defined maximum length
Enforcing schema compliance (e.g., JSON with specific fields only)
Blocking inputs containing disallowed patterns (e.g., ignore previous instructions, system override)
Restricting input sources to authenticated or trusted origins
Ensuring prompts match an approved task or intent category
Input Sanitization
Sanitization transforms input to remove, neutralize, or normalize unsafe elements while preserving legitimate intent. Sanitization ensures that the input is made safe before processing.
Examples of Sanitization
Normalizing Unicode to remove obfuscation (e.g., leetspeak, homoglyphs)
Stripping zero-width or invisible characters
Decoding and inspecting encoded content (Base64, hex) before use
Escaping or isolating untrusted text so it cannot be interpreted as instructions
Removing or redacting sensitive data (PII, secrets)
Technical Details
Failure to validate LLM output introduces several risks including:
Accidental or malicious execution of model-generated code (e.g., eval, exec, Function, subprocess calls)
Injection of harmful commands or payloads into tools, agents, or downstream applications
Execution of hallucinated instructions that modify resources, corrupt data, or trigger destructive operations
Leakage of internal or sensitive information through improperly filtered responses
Exploitation of agents that automatically convert LLM output into actions (“AI code injection”)
Loss of safety, explainability, reliability, and auditability in automated pipelines
Violations of governance, logging, and traceability requirements
Attack Vector: LLM output → downstream interpreter / agent tool
Attack Complexity: Low (LLM can be tricked into generating dangerous primitives)
Privileges Required: None
User Interaction: None (fully autonomous execution paths are most at risk)
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium to High
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Do not allow malicious content via hidden prompts written in leetspeak
AI_APP_SEC_032
Violation Summary Allowing prompts written in leetspeak (e.g., h4x0r, 3v4l, 1nj3ct, byp4ss) or similar obfuscated language enables attackers to evade input validation, safety filters, and policy enforcement mechanisms. Leetspeak transforms malicious intent into visually altered but semantically equivalent text, allowing prompt injection, jailbreak attempts, encoded instructions, and policy bypasses to slip past keyword-based detection and moderation layers. This weakens the integrity and reliability of LLM-driven systems, especially in agentic and autonomous workflows.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details
Allowing leetspeak introduces several risks including:
Bypass of keyword-based safety, moderation, and policy filters
Injection of malicious or unsafe instructions disguised as benign input
Increased success of encoded or obfuscated prompt attacks
Manipulation of agent reasoning or tool invocation logic
Reduced auditability and explainability due to obfuscated intent
Amplification of downstream risks when leetspeak-generated outputs drive actions
Attack Vector: LLM input prompt (user, agent, or external system)
Attack Complexity: Low
Privileges Required: None
User Interaction: None (especially in automated or agent-driven scenarios)
Confidentiality Impact: High
Integrity Impact: High to Critical
Availability Impact: Low
Framework
CERT-In AI Blueprint 2026
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
MCP server must not interact directly with an LLM
AI_APP_SEC_033
Violation Summary When an MCP server directly interacts with an LLM—rather than operating only through an authenticated, validated, policy-enforcing MCP client—it collapses the trust boundary between system capabilities and untrusted model output. This allows an LLM to influence, manipulate, or trigger server-side actions without authorization or validation. Direct LLM-to-server interaction bypasses safety controls, authentication layers, input validation, output filtering, logging standards, and audit requirements. This exposes the environment to unbounded prompt injection, unsafe tool execution, data leakage, and full-system compromise.
Affected Assets
LLM
MCP Server
Severity
Critical
Technical Details Allowing an MCP server to directly interact with an LLM introduces several risks including:
Execution of unsafe, hallucinated, or malicious LLM-generated instructions on high-privilege server tools
Prompt injection attacks gaining direct access to server capabilities
Bypassing client-side authentication, authorization, validation, and logging layers
Data leakage through uncontrolled LLM requests or responses
Inability to enforce least-privilege and zero-trust boundaries between model and system operations
Loss of auditability because actions occur without the MCP client as an intermediary
Violations of governance, safety, and regulatory obligations for high-risk AI systems
Attack Vector: LLM output → server action path
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium to High
Framework
OWASP-LLM-Top10
NIST AI RMF
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Clear exit or termination criteria must exist for the agent to consider its task complete and stop executing
AI_APP_SEC_034
Violation Summary When an AI agent is not given explicit, enforceable exit or termination criteria, it may continue executing indefinitely, escalate actions beyond intended scope, repeatedly invoke tools, consume excessive compute, or enter unsafe operational loops. Lack of defined stopping conditions increases the risk of runaway behavior, unintended system modifications, resource exhaustion, privacy violations, and unbounded interaction with external systems or MCP tools. Agents without termination logic become unpredictable, ungovernable, and potentially harmful.
Affected Assets
AI Agent
Severity
High/Critical
Technical Details Missing termination criteria introduces several risks including:
Infinite or runaway task execution that triggers unnecessary or harmful actions
Repeated tool invocation (MCP or external APIs), leading to data exposure or workflow corruption
Accidental escalation of privileges as the agent searches endlessly for ways to complete the task
Hallucination-driven decisions due to self-reinforcing reasoning loops
Excessive resource consumption or uncontrolled cost
Increased attack surface for prompt injection that pushes the agent into unsafe recursive behavior
Violations of safety, oversight, and accountability requirements
Attack Vector: Agent reasoning cycle / task execution loop
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Medium to High (depending on tool access)
Integrity Impact: High
Availability Impact: Medium to High
Framework
OWASP-LLM-Top10
NIST AI RMF
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Agents must log all interactions with an LLM
AI_APP_SEC_035
Violation Summary If agents do not log their interactions with an LLM, including prompts, responses, tool requests, and reasoning triggers, organizations lose visibility into how decisions were made, what data was exchanged, and whether harmful or unauthorized actions occurred. Missing LLM interaction logs break auditability, hinder incident response, obscure the source of incorrect or unsafe outputs, and prevent compliance verification. Lack of logging also enables attackers to exploit the agent–LLM channel without detection.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details Not logging agent ↔LLM interactions introduces several risks including:
Inability to reconstruct how the agent reached a decision or triggered an action
Loss of forensic evidence needed for incident response or regulatory review
Undetected prompt injection, harmful outputs, or unsafe tool invocations
Unmonitored leakage of sensitive data or PII through prompts or responses
Difficulty identifying hallucination-driven failures or behavioral drift
Loss of traceability required for governance, transparency, and safety assurance
Violations of logging, documentation, and accountability requirements
Attack Vector: Agent ↔ LLM communication channel
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
AI Threats and Exploits
Compliance
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
The AI Model must validate and sanitize any input before processing
AI_APP_SEC_038
Violation Summary
When AI Model accepts input without proper validation and sanitization, it becomes highly susceptible to prompt injection, encoded or hidden instructions, malicious payloads, and adversarial manipulation. Unsanitized inputs—originating from users, agents, tools, MCP servers, or external systems—can override system instructions, bypass guardrails, contaminate reasoning, and trigger unsafe downstream actions. This risk is amplified in agentic and tool-enabled environments where AI Model output directly influences real systems.
Affected Assets
LLM
Severity
Critical
Input Validation
Validation ensures that input conforms to expected structure, type, length, format, and policy constraints before it is processed by the AI Model. It answers the question if the input is allowed to be processed.
Examples of Validation
Rejecting prompts longer than a defined maximum length
Enforcing schema compliance (e.g., JSON with specific fields only)
Blocking inputs containing disallowed patterns (e.g., ignore previous instructions, system override)
Restricting input sources to authenticated or trusted origins
Ensuring prompts match an approved task or intent category
Input Sanitization
Sanitization transforms input to remove, neutralize, or normalize unsafe elements while preserving legitimate intent. Sanitization ensures that the input is made safe before processing.
Examples of Sanitization
Normalizing Unicode to remove obfuscation (e.g., leetspeak, homoglyphs)
Stripping zero-width or invisible characters
Decoding and inspecting encoded content (Base64, hex) before use
Escaping or isolating untrusted text so it cannot be interpreted as instructions
Removing or redacting sensitive data (PII, secrets)
Technical Details
Failure to validate and sanitize AI Model input introduces several risks including:
Prompt injection that overrides system and developer intent
Encoded, obfuscated, or hidden instructions bypassing safety controls
Injection of malicious content that manipulates tool usage or agent behavior
Leakage of sensitive data caused by adversarial prompt construction
Hallucination amplification driven by malformed or hostile inputs
Propagation of unsafe or untrusted instructions to downstream systems
Loss of transparency, auditability, and policy enforcement across AI workflows
Attack Vector: AI Model input channel (user, agent, tool, MCP, external system)
Attack Complexity: Low
Privileges Required: None
User Interaction: None (especially in autonomous or agent-driven flows)
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Sanitize and validate all input to the AI Model
AI_APP_SEC_039
Violation Summary If input sent to an LLM is not validated and sanitized, the system becomes vulnerable to prompt injection, obfuscated or encoded instructions, malformed payloads, and adversarial manipulation. Unchecked inputs originating from users, agents, tools, uploaded files, MCP services, or external systems can override system intent, bypass safety controls, contaminate reasoning, and trigger unsafe downstream actions. This risk is amplified in agentic, tool-enabled, and autonomous workflows where LLM output directly influences real systems.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details
Failure to validate and sanitize input before sending it to an LLM introduces several risks including:
Prompt injection that overrides system and developer intent
Encoded, hidden, or obfuscated instructions (Base64, leetspeak, zero-width characters) bypassing guardrails
Injection of malicious content that manipulates tool usage or agent behavior
Leakage of sensitive data caused by adversarial prompt construction
Hallucination amplification driven by malformed or hostile inputs
Propagation of unsafe or untrusted instructions to downstream systems
Loss of transparency, auditability, and policy enforcement across AI workflows
Attack Vector: Input channel → LLM (user, agent, tool, file ingestion, MCP, external system)
Attack Complexity: Low
Privileges Required: None
User Interaction: None (especially in autonomous or agent-driven flows)
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Do not allow malicious content via prompts included in uploaded files
AI_APP_SEC_040
Violation Summary If uploaded files (documents, PDFs, spreadsheets, images with OCR, code files, logs) are ingested by an LLM or agent without inspection for malicious prompts, attackers can embed hidden, encoded, or context-manipulating instructions that influence model behavior. These “prompt-in-files” attacks allow adversaries to bypass input controls, poison agent reasoning, extract sensitive data, or trigger unauthorized tool actions—often without any visible user prompt.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Failing to scan uploaded files for malicious prompts introduces several risks including:
Hidden prompt injection embedded in document text, comments, metadata, or OCR layers
Encoded or obfuscated instructions (Base64, leetspeak, zero-width characters) evading detection Cross-context contamination where file content overrides system or developer instructions
Unauthorized tool invocation or workflow manipulation driven by file-based prompts
Leakage of sensitive data due to adversarial instructions embedded in files
Loss of explainability when behavior is influenced by unseen file content
Violations of transparency, auditability, and policy enforcement requirements
Attack Vector: File upload > document ingestion / OCR / parsing pipeline
Attack Complexity: Low
Privileges Required: None
User Interaction: None, especially in automated ingestion or agent workflows.
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Do not allow prompts that can execute malicious commands at runtime
AI_APP_SEC_059
Violation Summary If prompts are not checked for content that may trigger runtime command execution, attackers can inject malicious instructions that cause the system to execute shell commands, file operations, or other high-privilege actions. In agentic or tool-enabled environments, LLM outputs can directly influence interpreters, orchestration engines, or automation pipelines. Without execution safeguards, prompt injection can escalate from text manipulation to full system compromise.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Failing to enforce checks on prompts that may execute commands introduces several risks including:
Command injection through prompts that trigger shell operations
Execution of malicious primitives such as eval, exec, subprocess calls, or dynamic interpreters
Corruption of files, configurations, or operational state
Loss of integrity and trust in AI-driven automation
Violations of secure coding, governance, and runtime safety controls
Attack Vector: Prompt → runtime interpreter
Attack Complexity: Low
Privileges Required: None
User Interaction: None (especially in autonomous or agent-driven workflows)
Confidentiality Impact: Critical
Integrity Impact: Critical
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/
Enforce synthetic content provenance, labeling, and watermarking for AI-generated outputs
AI_APP_SEC_064
Violation Summary
AI systems that generate text, images, audio, video, or code without provenance metadata, content labels, or embedded watermarks enable disinformation, impersonation, and undetectable forgery. AI risk management transparency and accountability expectations and content authenticity guidance require disclosed origin, machine-readable provenance, and tamper-evident markers on all synthetic outputs.
Affected Assets
AI Model
AI Agent
LLM
Severity
Critical
Technical Details
Missing or weak synthetic content provenance, labeling, and watermarking for ai-generated outputs controls introduces risks including:
Returning AI-generated content without machine-readable provenance metadata, such as model identifiers, timestamps, or content origin tags.
Serving AI-generated content without a clear label indicating its synthetic origin.
Producing AI-generated images, audio, or video without embedding a steganographic or cryptographic watermark.
Utilizing unsigned provenance metadata, which allows the origin information to be silently removed or tampered with.
Accepting externally supplied content claimed as human-authored without verifying the absence of AI provenance markers upon ingestion.
Failing open when labeling or watermarking errors occur, resulting in the delivery of unlabeled synthetic content instead of safely blocking the request.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
California SB 942
China CAC GenAI Measures
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
South Korea AI Framework Act
References
https://genai.owasp.org/llm-top-10/
Do not allow prompts with high-risk system commands
AI_APP_SEC_066
Violation Summary If prompts are not inspected for high-risk system or shell commands, attackers can embed instructions that lead to execution of destructive or unauthorized operations. In agentic and tool-enabled environments, LLM outputs may be interpreted as executable instructions, allowing prompt content to directly trigger command execution. Without detection and filtering of high-risk commands, prompt injection can escalate into command execution, system compromise, data destruction, or unauthorized access.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Failing to check prompts for high-risk system and shell commands introduces several risks including:
Execution of dangerous commands such as file deletion, privilege escalation, or system modification
Command injection through prompts interpreted by shell, scripting engines, or automation tools
Unauthorized access to system resources, environment variables, or sensitive files
Data exfiltration via command-line utilities or scripted API calls
Lateral movement across systems using injected commands
Corruption or deletion of critical data and configurations
Abuse of automation pipelines and agent toolchains
Loss of integrity, reliability, and control over AI-driven operations
Attack Vector: Prompt → shell / system interpreter / agent tool execution
Attack Complexity: Low
Privileges Required: None
User Interaction: None (especially in automated or agent-driven workflows)
Confidentiality Impact: Critical
Integrity Impact: Critical
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Detect direct string interpolation of untrusted input into LLM prompts
AI_APP_SEC_067
Violation Summary
Direct embedding of unvalidated, unsensitized, or unescaped user-controlled data into LLM prompt strings via f-strings, str.format(), string concatenation, or template substitution allows adversaries to inject instructions that override system context, hijack agent behavior, or exfiltrate sensitive information. Code must enforce structural separation between prompt templates and untrusted data so that user input is never interpreted as instructions by the model.
Affected Assets
AI Agent
MCP Server
AI Model
Severity
Critical
Technical Details Direct string interpolation introduces several risks including:
Undetectable prompt injection, unpredictable, unsafe or incorrect output
Bypass safety and governance controls
Unsafe or inconsistent agent behavior, data exfiltration and, system context overrides
Attack Vector: Prompt
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
CERT-In AI Blueprint 2026
EU AI Act
MITRE ATLAS
NIST AI RMF
OWASP-LLM-Top10
References
Detect LLM output used directly in security-sensitive decisions without Human-in-the-Loop (HITL) validation
AI_APP_SEC_068
Violation Summary
Detects and prevents the direct use of unvalidated LLM outputs in security-sensitive decisions without Human-in-the-Loop (HITL) validation. LLM output is probabilistic and may hallucinate or be adversarially manipulated, so it must never be the sole authority for a security decision.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
Critical
Technical Details
Missing or weak Human-in-the-Loop (HITL) validation or rule-based check controls introduces risks including:
Relying on raw LLM output as the sole condition in an access, authentication, or permission check without a HITL gate or rule-based check.
Feeding LLM output directly into a function enforcing security policy or classification without a HITL gate or rule-based check.
Granting elevated privileges or expanded scope based on LLM output without a HITL gate or rule-based check.
Failing to implement a deterministic validation step (like an allowlist or schema enforcement) between the LLM response and a security-sensitive operation.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: None to Low
User Interaction: None to Limited
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
California SB 942
Colorado SB 26-189
EU AI Act
MITRE ATLAS
NIST AI RMF
OWASP-LLM-Top10
South Korea AI Framework Act
References
https://genai.owasp.org/llm-top-10/
AI Agent must implement Human-in-the-Loop (HITL) approval flow for risky operations like delete, purge, destroy
AI_APP_SEC_069
Violation Summary If high-risk operations (e.g., delete, purge, destroy, move, export) are executed without Human-in-the-Loop (HITL) oversight, AI agents and automated systems can perform destructive or unauthorized actions without validation. In agentic and tool-enabled environments, LLM outputs can directly trigger these operations, and prompt injection, hallucination, or misinterpretation can lead to irreversible damage. Absence of HITL removes a critical control layer needed to prevent catastrophic system, data, or business impact.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Not implementing HITL for high-risk operations introduces several risks including:
Accidental or malicious deletion, purge, or destruction of critical data or resources
Execution of irreversible actions triggered by prompt injection or adversarial inputs
Hallucination-driven decisions resulting in destructive operations
Unauthorized changes due to compromised agents or misconfigured permissions
Inability to detect or stop harmful actions before execution
Lack of accountability and validation for critical system changes
Increased blast radius in automated workflows without human checkpoint
Violations of governance, risk management, and safety requirements
Attack Vector: LLM output → agent tool execution / automation pipeline
Attack Complexity: Low
Privileges Required: None
User Interaction: None (in absence of HITL controls)
Confidentiality Impact: Medium
Integrity Impact: Critical
Availability Impact: Critical
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Detect and block all forms of prompt injection attacks in user inputs and file contents
AI_APP_SEC_070
Violation Summary
Failure to detect and block prompt injection attacks in user inputs, uploaded files, retrieved content, emails, web pages, or other external context sources can allow attackers to manipulate LLM behavior, override system instructions, bypass safety controls, exfiltrate sensitive data, trigger unauthorized tool execution, and compromise autonomous agent decision-making. Prompt injection is recognized by industry and government guidance as one of the most critical security risks affecting generative AI and agentic systems.
Affected Assets
AI Model
AI Agent
Severity
Critical
Technical Details
Prompt injection attacks exploit the inability of LLMs to reliably distinguish trusted instructions from untrusted external content. Malicious instructions may be embedded directly in prompts or indirectly within files, documents, web pages, emails, images, repositories, or retrieved content.
Failure to detect and block such attacks can result in:
Override or suppression of system instructions
Leakage of sensitive or confidential data
Unauthorized tool invocation or API execution
Execution of destructive or unsafe operations
Manipulation of autonomous agent reasoning
Cross-context or cross-agent contamination
Safety guardrail bypass and policy evasion
Credential theft or token exposure
Privilege escalation through indirect instructions
Compromise of downstream systems and workflows
Corruption of outputs, decisions, or generated code
Persistent propagation across multi-agent environments
Attack Vector: Prompt / Context Injection
Attack Complexity: Low to Medium
Privileges Required: None
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/?utm_source=chatgpt.com
https://genai.owasp.org/initiatives/agentic-security-initiative/?utm_source=chatgpt.com
https://artificialintelligenceact.eu/ai-act-explorer/?utm_source=chatgpt.com
https://www.nist.gov/itl/ai-risk-management-framework?utm_source=chatgpt.com
Enforce chemical, biological, radiological, or nuclear (CBRN) threat prevention safeguards in AI-enabled systems
AI_APP_SEC_071
Violation Summary
AI systems that generate, design, or analyze chemical, biological, radiological, or nuclear (CBRN) content without screening and biological safeguards can enable misuse for weapons development, regulated-pathogen work, or other high-consequence harm.
Affected Assets
LLM
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak CBRN controls in deployment code introduces risks including:
Unscreened nucleic acid synthesis, lab routing, or bioinformatics uploads to external providers
CBRN-capable model, agent, or scientific-tool paths without classifiers, policy gates, or tool allowlists
Exposure of pathogen, clinical, or dual-use biological data without authentication, least privilege, labels, or encryption
Fail-open behavior when screening fails, times out, is ambiguous, or is bypassed
Soft handling (log or warn only) of high-confidence screening hits while the pipeline continues
Client-controlled flags or parameters that disable hazardous-sequence screening or biosafety filters
Attack Vector: Network
Attack Complexity: Medium
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
EO 14110
EU AI Act
NIST AI RMF
References
https://www.nist.gov/itl/ai-risk-management-framework/ai-rmf-1-0
AI systems making consequential decisions must implement explainability mechanisms disclosing decision factors
AI_APP_SEC_072
Violation Summary
AI systems used for consequential decisions such as employment, credit, health, legal, or financial determinations must generate and expose explanations of the key factors influencing each decision. Without explainability mechanisms, affected individuals cannot understand or challenge AI-driven outcomes, violating transparency requirements across multiple jurisdictions.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak explainability mechanisms for ai systems making consequential decisions introduces risks including:
AI model inference call used for consequential decisions (employment, credit, health, legal, financial) that does not generate or return an explanation of the key factors influencing the decision alongside the prediction or recommendation.
Consequential AI prediction pipelines that lack integration with an explainability method (SHAP, LIME, feature importance, attention weights, saliency maps, or equivalent) or do not expose attribution data to downstream consumers.
Returns or sends a consequential AI decision to a user, API consumer, or downstream system without including a human-readable explanation of the decision factors.
Discards, filters, or omits explainability output before delivering a consequential AI decision result.
Model serving or inference config files for consequential AI use cases that lack an explainability mode, explanation format, or attribution method field.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
California SB 942
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
Japan AI Basic Plan
NIST AI RMF
South Korea AI Framework Act
Vietnam AI Law
References
https://www.nist.gov/itl/ai-rmf
https://leginfo.legislature.ca.gov/
AI consequential decisions must include a human review pathway before final determination
AI_APP_SEC_073
Violation Summary
AI systems making consequential decisions in employment, credit, health, legal, or financial domains must provide a human review pathway so that no wholly automated decision with material impact on an individual proceeds without the option for human oversight. Absence of human review mechanisms undermines accountability and denies affected persons the right to challenge automated outcomes.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak human review pathway controls for ai consequential decisions introduces risks including:
Produces a final consequential AI decision (hire/reject, approve/deny, diagnose, sentence, grant/revoke) and commits or returns it without a preceding check for human review status, approval flag, or reviewer sign-off.
Writes a consequential AI decision to a database, queue, API response, or notification channel without an intermediate human review step or approval gate.
Consequential AI decision handlers that lack a mechanism to flag the decision for human review, queue it for manual override, or expose an appeal or contest endpoint to the affected individual.
Skips, disables, or short-circuits a human review check based on a flag, configuration toggle, or threshold, allowing a consequential decision to proceed without review.
Consequential decision pipelines that lack a log entry or audit record capturing whether human review occurred, the reviewer identity, and the review outcome.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
California SB 942
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
Japan AI Basic Plan
NIST AI RMF
South Korea AI Framework Act
Vietnam AI Law
References
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
AI systems must implement emergency shutdown and forced-termination mechanisms
AI_APP_SEC_074
Violation Summary
AI systems deployed in production must implement emergency shutdown and forced-termination capabilities that allow operators to immediately halt AI operations when the system behaves unexpectedly, causes harm, or poses safety risks. Without kill-switch mechanisms, malfunctioning AI systems cannot be stopped in time to prevent escalating damage.
Affected Assets
AI Model
AI Agent
LLM Application
MCP Server
Severity
Critical
Technical Details
Missing or weak emergency shutdown and forced-termination mechanisms for ai systems introduces risks including:
A long-running process (serve_forever, app.run, uvicorn.run, while-True entry loop) without registering SIGTERM/SIGINT handlers and without an admin shutdown endpoint in the server class.
A long-running server call (serve_forever, app.run) lacks shutdown logic to stop new requests, drain in-flight work, and release resources.
Shutdown handlers lack a hard timeout that force-terminates when graceful shutdown exceeds a time limit. If no handler exists, flag the server start call.
Agent function or while-True loop that lacks an iteration cap, wall-clock timeout, or error-rate breaker. Exclude the main server or entrypoint loop already flagged by rule 1.
Sets signals to SIG_IGN, catches and discards termination signals, or suppresses kill-switch triggers.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
EU AI Act
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
References
Detect and block use of facial recognition APIs or libraries in employment decision workflows
AI_APP_SEC_075
Violation Summary
Code must not use facial recognition technology — including facial analysis, emotion detection, or biometric identification via facial features — within employment decision workflows such as hiring, promotion, termination, or performance evaluation. Use of facial recognition in these contexts introduces bias, privacy violations, and discriminatory impact prohibited by law.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak controls to detect and block use of facial recognition in employment decision workflows introduces risks including:
Imports of facial recognition libraries (face_recognition, deepface, insightface, dlib, OpenCV face cascade/DNN, mediapipe face) or cloud APIs (AWS Rekognition, Azure Face, Google Vision face detection) in code related to hiring, recruitment, HR screening, performance evaluation, promotion, or termination.
Captures, processes, or analyzes facial images or video of candidates or employees and uses the results to score, rank, filter, or influence an employment decision.
Applies emotion recognition or behavioral analysis based on facial expressions within an employment or HR context.
Compares facial biometric templates or embeddings to make or inform employment decisions such as attendance-based discipline, performance tracking, or interview scoring.
Passes facial analysis outputs, face embeddings, or recognition scores as input to functions that produce employment decisions.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: None
Framework
EU AI Act
ISO/IEC 42001
Maryland HB 1202
References
High-risk AI systems must provide enhanced disclosures including decision factors, known limitations, accuracy metrics, and appeal rights
AI_APP_SEC_076
Violation Summary
AI systems classified as high-risk must deliver enhanced transparency disclosures to affected individuals, including the key factors influencing each decision, known system limitations and potential biases, accuracy and performance metrics, and clear instructions for exercising appeal or human review rights. Standard disclosure alone is insufficient for high-risk deployments where decisions carry legal or similarly significant effects.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak enhanced disclosure controls for high-risk ai systems introduces risks including:
High-risk AI system response handler or output function that delivers a decision to the affected individual without including the key factors or features that influenced the decision.
High-risk AI system configurations, model cards, or response templates that do not disclose known system limitations, potential biases, or conditions under which the system may produce unreliable results.
High-risk AI system documentation, configuration, or response metadata that does not include performance metrics such as accuracy rate, precision, recall, false positive rate, or equivalent quality indicators.
High-risk AI decision responses that do not include instructions for the affected individual to request human review, appeal the decision, correct inaccurate input data, or opt for human-only decision-making.
AI systems that declare a high risk classification but deliver responses using the same disclosure template as low or medium risk systems, without additional decision factor, limitation, metric, or appeal information.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
California SB 942
EU AI Act
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
Vietnam AI Law
References
https://leginfo.legislature.ca.gov/
https://www.digitaltransformationoffice.gov.au/
AI systems must provide consumer notice before using ADMT in a consequential decision
AI_APP_SEC_077
Violation Summary
Any code path that executes a consequential AI decision (loan scoring, hiring filters, benefits eligibility, housing determinations, insurance underwriting, healthcare triage) must invoke a consumer disclosure mechanism before the decision runs. The disclosure must inform the consumer that automated decision-making technology is being used and provide instructions for how the consumer can obtain additional information. Without this pre-interaction notice, organizations violate transparency obligations under automated decision-making regulations and expose consumers to undisclosed algorithmic outcomes.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak consumer notice controls before using admt in consequential decisions introduces risks including:
Invokes an AI model, scoring API, or decision pipeline for a consequential domain (employment, credit, housing, insurance, healthcare, education, government benefits) without a preceding call to a disclosure, notice, or notification function that informs the consumer that automated decision-making technology is being used.
Skips or short-circuits the consumer disclosure step via a flag, feature toggle, or early-return condition before reaching the consequential decision execution.
AI decision entry points (controllers, service methods, API handlers) for consequential domains that have no reference to any disclosure, notice, consent, or ADMT-use-notification function anywhere in their call chain.
Consumer notice calls that do not record the disclosure event (timestamp, consumer identifier, decision type) to an audit log or persistent store, making it impossible to demonstrate that notice was provided.
Disclosure implementations that notify the consumer of ADMT use but do not include or reference a channel, URL, or contact method through which the consumer can request additional information about the system.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
California SB 942
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
Vietnam AI Law
References
https://leg.colorado.gov/bills/sb26-189
Human review workflows for AI consequential decisions must enforce override authority, reviewer context, and no auto-approve defaults
AI_APP_SEC_078
Violation Summary
Human review implementations in AI consequential decision systems must satisfy three enforceable properties: the reviewer role must carry an explicit override or modify permission on the decision outcome; the review payload presented to the reviewer must include the required regulatory context fields (intended use, material limitations, input categories, principal decision factors); and no timeout or fallback logic may auto-approve or default to the AI system's output if the reviewer takes no action. Without these controls, human review is a procedural formality rather than a meaningful check, violating the definition of meaningful human review under automated decision-making regulations.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak human review workflow enforcement controls for ai consequential decisions introduces risks including:
Human review workflow code where the reviewer role, permission set, or access control definition does not include an explicit override, modify, or reject permission on the AI decision outcome. Flag roles that are read-only or advisory without a write or decision-modification scope.
Human review handlers, API responses, or UI data structures that present a consequential AI decision for review without including all of the following fields: intended_use (or equivalent describing the AI system's purpose), limitations (or known_limitations), input_categories (or data_types_used), and decision_factors (or principal_factors or explanation).
Human review workflow that includes a timeout, expiry, or scheduled job that automatically approves, confirms, or forwards the AI system's output as the final decision if the reviewer does not act within a defined period.
Path in a human review workflow that falls back to accepting the AI system's recommendation as the final decision without explicit reviewer action, including fallbacks triggered by errors, exceptions, or reviewer unavailability.
Human review completions (approve, modify, or reject actions) that are not recorded to an audit log with the reviewer identity, the action taken, the timestamp, and the final decision value, making the review unverifiable.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
California SB 942
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
References
https://leg.colorado.gov/bills/sb26-189
https://leginfo.legislature.ca.gov/
Enforce rate limiting and throttling on AI API calls
AI_APP_SEC_079
Violation Summary
AI API calls (foundation model invocations, LLM completions, embedding requests, image/audio generation, and proxied user-facing AI endpoints) without rate limiting and throttling are vulnerable to denial-of-service, runaway cost, model-tier exhaustion, and abuse via prompt floods or unbounded agent loops. Code must enforce per-principal rate limits, token and cost budgets, provider 429/backoff handling, and route-level throttling on every AI API call site so that no single user, tenant, or agent loop can exhaust the service or the upstream model quota.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak rate limiting and throttling controls on ai api calls introduces risks including:
Invocation of a foundation model client (openai, anthropic, google.generativeai, cohere, mistral, bedrock InvokeModel, vertexai, ollama, transformers pipeline) not preceded by a token-bucket acquire, semaphore acquire, rate-limit middleware, or rate-limit decorator on the same function or call site.
Rate-limit logic that uses a single global counter or shared bucket without keying on user id, API key, tenant id, IP, or session id.
For or while loops that call an AI API inside the loop body without a maximum iteration counter and without a sleep, throttle, or rate-limit acquire between calls.
AI API call sites that pass max_tokens, input length, or batch size derived from external or user input without a hard upper-bound check immediately preceding the call.
AI API call sites with no try/except capturing provider rate-limit errors (HTTP 429, RateLimitError, ResourceExhausted, TooManyRequests) with exponential backoff, or that retry inside a tight loop with no delay.
HTTP or RPC handlers (Flask, FastAPI, Express, gRPC) that proxy to an AI model without a rate-limit middleware or decorator applied to the route.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: High
Framework
OWASP-LLM-Top10
NIST AI RMF
CERT-In AI Blueprint 2026
References
https://genai.owasp.org/llm-top-10/
https://www.nist.gov/itl/ai-rmf
https://www.cert-in.org.in/
Data Security and Privacy
Protects PII, prevents data leakage, and enforces privacy controls across AI models and agents.
Do not store secrets in code
AI_DAT_SEC_001
Violation Summary Storing secrets, such as API keys, access tokens, service credentials, MCP tokens, encryption keys, or database passwords, directly in code, configuration files, or agent prompt templates introduces an immediate and critical security vulnerability. Hard-coded secrets are easily exposed through source control, logs, error messages, LLM interactions, and dependency analysis. Once leaked, these credentials can be used to impersonate services, manipulate AI agent behavior, exfiltrate data, or compromise entire environments.
Affected Assets
LLM
AI Agent
MCP Server
Severity
Critical
Technical Details Storing secrets in code introduces several risks including:
Unauthorized access to internal systems, APIs, and third-party services
Full environment compromise if privileged keys (e.g., root tokens) are exposed
Impersonation of agents, MCP clients, or downstream services
Lateral movement enabled through leaked credentials
Leakage via LLM outputs, agent error messages, or repository scans
Irreversible compromise of production systems due to difficult secret rotation
Violations of governance, transparency, and credential-handling requirements
Attack Vector: Source code / repository / agent configuration Attack Complexity: Low Privileges Required: None User Interaction: None Confidentiality Impact: Critical Integrity Impact: Critical Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
If PII data must be shared, it must be encrypted
AI_DAT_SEC_009
Violation Summary Transmitting personally identifiable information (PII) without encryption exposes sensitive user data to interception, tampering, unauthorized access, and regulatory non-compliance. Unencrypted PII flowing between AI agents, MCP clients and servers, microservices, or external APIs can be harvested by attackers or internal adversaries through network sniffing, logging systems, or compromised intermediaries. Such exposure creates severe privacy, legal, operational, and reputational risks.
Affected Assets
LLM
AI Agent
MCP Server
Severity
Critical
Technical Details Transmitting unencrypted PII introduces several risks including:
Exposure of sensitive user information through network interception
Unauthorized access to identity data, enabling fraud or impersonation
Regulatory violations (GDPR, EU AI Act, state privacy laws)
Inability to ensure integrity or authenticity of transmitted data
Leakage through LLM logs, telemetry, or debugging outputs
Lateral movement or privilege escalation through harvested identity data
Failure to meet encryption, security, and risk-management obligations
Attack Vector: Network transit / API calls / agent communication Attack Complexity: Low Privileges Required: None User Interaction: None Confidentiality Impact: Critical Integrity Impact: High Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Do not log PII
AI_DAT_SEC_010
Violation Summary Logging personally identifiable information (PII) exposes sensitive user data to unauthorized access, replication, and long-term retention in unsecured or low-visibility systems. Log files are frequently accessible to broader engineering, operations, analytics, or third-party tools and often persist indefinitely. Once PII enters logs, it becomes extremely difficult to control, delete, audit, or protect—creating severe privacy, compliance, and security risks across all AI and MCP-enabled environments.
Affected Assets
LLM
AI Agent
MCP Server
Severity
Critical
Technical Details
Logging PII introduces several risks including:
Unauthorized internal access or external compromise of sensitive information
Accidental disclosure through debugging tools, telemetry pipelines, or log aggregators
Persistent exposure that violates data minimization and retention requirements
Inability to satisfy deletion, correction, or subject rights requests
Propagation of PII through downstream systems (LLM training data, observability tools, backups)
Legal and regulatory violations under GDPR, state privacy laws, and the EU AI Act
Loss of trust and reputational damage due to preventable data leakage
Attack Vector: Logging systems / telemetry pipelines / observability tooling
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: Medium
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
IRAP
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Do not send PII to AI Models
AI_DAT_SEC_011
Violation Summary Sending personally identifiable information (PII) to an AI Model exposes sensitive data to uncontrolled processing, persistence, training retention, unauthorized internal access, and unintended disclosure. AI Models are not guaranteed to handle PII according to data-minimization or privacy-by-design principles, and model outputs may inadvertently reveal, transform, or propagate sensitive information. This creates severe privacy, regulatory, and security risks across all AI-driven workflows.
Affected Assets
AI Agent
Severity
Critical
Technical Details Sending PII to an AI Model introduces several risks including:
Leakage of sensitive user information through outputs or indirect inference
Inclusion of PII in model logs, telemetry, or monitoring systems
Potential model retention or memorization of PII, enabling future extraction
Non-compliance with privacy regulations due to uncontrolled third-party processing
Exposure through prompt injection attacks that pull stored or inferred PII
Inability to enforce deletion, consent, or data subject rights
Violations of transparency, purpose limitation, and privacy-by-design obligations
Attack Vector: AI Model input channel
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: Medium
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Mask PII on user interfaces
AI_DAT_SEC_012
Violation Summary Displaying unmasked personally identifiable information (PII) on user interfaces exposes sensitive data to unauthorized viewing, shoulder surfing, screen sharing leaks, and over-privileged internal access. Any UI that renders full PII—names, addresses, SSNs, phone numbers, emails, financial data, or identifiers—creates a high risk of accidental disclosure and non-compliance. Unmasked PII can also be captured in screenshots, monitoring tools, session replay systems, or logs, further amplifying exposure.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details Not masking PII on UI introduces several risks including:
Unauthorized access or accidental exposure of sensitive identity information
Violation of least-privilege and data-minimization principles
Increased likelihood of data leakage via screenshots, video recordings, demos, or shared sessions
Compromise through malicious insiders or overexposed customer support tools
Replication of PII into frontend logs, browser telemetry, or third-party analytics
Regulatory violations related to privacy, transparency, and secure data handling
Loss of user trust and potential legal liability
Attack Vector: User interface display layer
Attack Complexity: Low
Privileges Required: None (visual exposure requires only observation)
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: Low
Availability Impact: Low
Framework
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Redact PII from uploaded files
AI_DAT_SEC_023
Violation Summary If uploaded files (documents, PDFs, spreadsheets, images with OCR, logs, archives) are ingested without redacting personally identifiable information (PII), sensitive data can be unintentionally exposed, propagated, or retained across AI systems. Unredacted PII may be processed by LLMs, logged, cached, embedded in prompts, or transmitted to external services—creating severe privacy, regulatory, and security risks that are difficult to detect and remediate after ingestion.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Failing to redact PII from uploaded files introduces several risks including:
Exposure of sensitive personal data through LLM processing, outputs, or logs
Propagation of PII into prompts, embeddings, vector stores, and downstream systems
Accidental disclosure via summaries, citations, or extracted insights
Inability to honor data minimization, retention limits, or subject rights requests
Increased blast radius when files are shared across agents or external tools
Elevated risk of data leakage through prompt injection or model inference
Violations of privacy-by-design, transparency, and record-keeping requirements
Attack Vector: File upload → parsing / OCR / document ingestion pipeline
Attack Complexity: Low
Privileges Required: None
User Interaction: None, especially in automated ingestion or agent workflows.
Confidentiality Impact: Critical
Integrity Impact: Medium
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Uploaded files must not contain PII (Singapore)
AI_DAT_SEC_024
Violation Summary This applies to the information that is considered PII in Singapore. If files uploaded to an AI agent are ingested without redacting PII, sensitive data can be unintentionally exposed, propagated, retained, or disclosed through agent reasoning, LLM prompts, logs, embeddings, or downstream tool calls. Because AI agents often summarize, transform, store, and share file contents across systems, unredacted PII significantly amplifies privacy, compliance, and security risks and makes post-incident remediation extremely difficult.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Failing to redact PII from files uploaded to AI agents introduces several risks including:
Exposure of sensitive personal data through agent outputs, summaries, or citations
Propagation of PII into prompts, vector databases, caches, logs, and telemetry
Uncontrolled sharing of PII with external LLM providers or third-party tools
Inability to comply with data minimization, retention limits, or deletion requests
Increased risk of data leakage via prompt injection or inference attacks
Broader blast radius when agents reuse or redistribute file content
Violations of privacy-by-design, transparency, and record-keeping requirements
Attack Vector: File upload → agent ingestion / parsing Attack Complexity: Low Privileges Required: None User Interaction: None (especially in automated or agent-driven workflows) Confidentiality Impact: Critical Integrity Impact: Medium Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
Singapore Personal Data Protection Act
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
No file should contain any PII
AI_DAT_SEC_025
Violation Summary If files contain unredacted personally identifiable information (PII), there is risk of sensitive data access to unrestricted access, misuse, and downstream propagation. This increases the likelihood of data leakage, unauthorized scraping, AI training contamination, and regulatory violations. Once exposed, PII can be copied, indexed, cached, or redistributed beyond the organization’s control.
Affected Assets
AI Agent
Severity
Critical
Technical Details
Failing to redact PII from publicly accessible files introduces several risks including:
Unrestricted access to sensitive personal data by internal users, contractors, or the public
Mass data harvesting, scraping, or indexing by automated tools and AI systems
Propagation of PII into LLM prompts, embeddings, search indexes, and external datasets
Inability to enforce consent, purpose limitation, or access controls
Permanent exposure due to caching, backups, screenshots, or mirrors
Increased insider threat and accidental disclosure risk
Severe regulatory, legal, and reputational impact
Attack Vector: Public file access / shared repositories / collaboration platforms
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: Medium
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Enforce output data minimization for model, tool, and API responses
AI_DAT_SEC_027
Violation Summary
Adversaries and misconfigured pipelines can cause the system to return excessive internal, sensitive, or user-identifying data in completions, tool results, logs, or client payloads. Without minimization, PII, secrets, full records, and operational metadata leak through normal response paths.
Affected Assets
AI Model
AI Agent
LLM
MCP Server
Severity
Critical
Technical Details
Missing or weak output data minimization for model, tool, and API responses controls introduces risks including:
Forwarding entire database rows, user profiles, complete document bodies, or unbounded retrieval sets into model prompts or client responses when only a subset is required.
Exposing raw internal metadata, such as primary keys, session identifiers, stack traces, file paths, or hostnames, to end users without redaction.
Placing secrets, API keys, tokens, passwords, private keys, or connection strings into completion text, tool results, error messages, or logs exposed to clients.
Returning unredacted tool, MCP, or plugin results to the user or upstream LLM without field allowlists, leading to arbitrary keys appearing in final outputs.
Appending full conversation histories, memory dumps, or prior-session transcripts to new responses instead of using a minimal summary or targeted excerpt.
Failing open on minimization or redaction errors, resulting in the return of original unfiltered payloads to the client rather than blocking or returning a safe error message.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: None to Low
User Interaction: None
Confidentiality Impact: High
Integrity Impact: None to Low
Availability Impact: Low
Framework
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Enforce decision logging, audit trail, and forensic readiness for AI-driven actions
AI_DAT_SEC_029
Violation Summary
AI systems that make or influence decisions affecting people, data, or operations without immutable audit trails and forensic-ready logs create accountability gaps and obstruct incident response. AI risk management accountability, transparency, and post-deployment monitoring expectations require traceable, tamper-evident records of every consequential AI-driven action.
Affected Assets
AI Model
AI Agent
LLM
MCP Server
Severity
Critical
Technical Details
Missing or weak decision logging, audit trail, and forensic readiness for ai-driven actions controls introduces risks including:
Failing to log model identifiers, input hashes, outputs, and acting principals creates significant accountability gaps, making it impossible to trace consequential decisions back to specific origins.
Permitting mutable logs that can be updated, truncated, or deleted compromises the integrity of the audit trail, undermining trust and forensic investigations.
The lack of shared correlation identifiers across multi-step processes prevents administrators from reconstructing the full causal chain of events.
Operating without configured retention or log rotation policies can lead to compliance violations and unmanaged storage.
Catching logging exceptions silently without alerting or failing closed can result in permanent loss of audit data when the logging sink becomes unreachable.
Omitting essential forensic context, such as data lineage and retrieval metadata, makes it impossible to reproduce, explain, or thoroughly investigate AI-driven actions post-deployment.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: None to Low
User Interaction: None
Confidentiality Impact: Low
Integrity Impact: High
Availability Impact: None to Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
Colorado SB 26-189
EO 14110
EU AI Act
IRAP
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/
https://www.nist.gov/itl/ai-risk-management-framework
https://genai.owasp.org/initiatives/agentic-security-initiative/
Enforce minimum six-month log retention for high-risk AI systems
AI_DAT_SEC_030
Violation Summary
Operators of high-risk AI systems must preserve automatically generated logs for at least six months in tamper-evident storage and must not delete or rotate them early. Without that floor, regulators and internal governance lose the evidence needed to reconstruct incidents, prove compliance, and defend decisions made by models and agents over their full evaluation window.
Affected Assets
AI Model
AI Agent
LLM Application
MCP Server
Severity
Critical
Technical Details
Missing or weak six-month minimum log retention for high-risk AI systems controls introduces risks including:
Writing or routing AI logs to storage without an explicit retention period or TTL makes it impossible to prove that records will survive the minimum compliance window.
Retention settings, expirations, rotation schedules, or cleanup jobs shorter than 180 days silently discard inference, decision, and error history before the regulatory floor is met.
Unconditional deletion, purge, truncate, or overwrite paths can erase high-risk AI evidence even when records have not aged past six months.
Size- or count-capped rotation and time-based rotation without durable archival can drop rotated segments permanently instead of retaining them for the full 180 days.
High-risk code paths that perform inference, scoring, classification, or decisions without automatic lifetime event logging leave gaps where critical activity was never captured.
Lack of mechanisms for deployers to collect, store, and interpret auto-generated logs blocks independent oversight and slows breach or misuse investigations.
Storing retention-sensitive records in non-tamper-evident systems increases the chance that historical logs are altered or repudiated after the fact.
Drift between environments—where one stack honors 180 days and another does not—produces inconsistent audit posture across the same high-risk AI workload.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: Low
Integrity Impact: High
Availability Impact: None to Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
Colorado SB 26-189
EU AI Act
NIST AI RMF
Vietnam AI Law
References
Disclose data acquisition methods including permissions, licensing, and preprocessing steps
AI_DAT_SEC_032
Violation Summary
AI systems that acquire, download, or preprocess training data without documenting the acquisition method, licensing terms, and preprocessing transformations prevent stakeholders from verifying data legality, replicating results, and assessing data quality. Transparency and documentation requirements for AI systems expect co-located or imported metadata describing how data was obtained and what was done to it before use.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak data acquisition method disclosure controls introduces risks including:
Downloads, fetches, streams, or reads training data (requests.get, urllib, boto3.download_file, load_dataset, open, or equivalent) without a co-located or imported license identifier, SPDX expression, license variable, or licensing comment specifying the terms under which the data was acquired.
Dataset class definitions or dataset configuration files that lack a license, permissions, or access_method attribute, property, or config field describing how the data was obtained.
Preprocessing or transformation functions (normalize, tokenize, filter, augment, clean, encode, transform, or equivalent) that lack a docstring or structured comment describing what transformations are applied.
Data pipeline or preprocessing configuration files with a pipeline, preprocessing, or transforms section that list only function names or class names with no accompanying description of their effect or purpose.
YAML, JSON, or TOML configuration files with a dataset or training_data section that are missing a license, acquisition_method, or permissions field.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: Low
Integrity Impact: High
Availability Impact: None
Framework
California-2023-AB2013-Chaptered
China CAC GenAI Measures
EU AI Act
ISO/IEC 42001
NIST AI RMF
References
https://artificialintelligenceact.eu/
https://www.nist.gov/itl/ai-rmf
https://leginfo.legislature.ca.gov/
https://www.airuniversity.af.edu/CASI/
Patient's acknowledgment of AI-assisted care must be documented and retained before clinical AI use
AI_DAT_SEC_033
Violation Summary
Healthcare AI systems must capture, record, and persist patient acknowledgment or consent for AI-assisted care before invoking AI for clinical purposes. Consent records must include patient identifier, timestamp, and scope of AI use, and must be stored in a persistent data store accessible for audit.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
High
Technical Details
Missing or weak patient acknowledgment documentation controls for ai-assisted care introduces risks including:
Invokes an AI model for clinical purposes (diagnosis, treatment recommendation, imaging analysis, or clinical decision support) without a preceding check that a patient consent or acknowledgment record exists and is valid.
Consent or acknowledgment record creation that does not capture all of: patient identifier, timestamp, and the scope or description of AI use consented to.
Path where patient consent or acknowledgment is captured in memory or a temporary variable but not written to a persistent data store before the clinical AI call is made.
Consent capture or acknowledgment mechanism that lacks a log entry or audit record written to a durable store linking the patient identifier, consent timestamp, and AI system or use case identifier.
Allows a clinical AI invocation to proceed when a consent check returns false, null, or an error, rather than halting execution.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: None
Framework
OWASP-LLM-Top10
NIST AI RMF
References
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
https://www.hhs.gov/hipaa/index.html
https://aibasicact.kr/
Biometric data collection must be preceded by explicit, documented consent capture
AI_DAT_SEC_036
Violation Summary
AI systems that collect biometric data — including facial images, fingerprints, voiceprints, iris scans, palm prints, gait patterns, or other physiological or behavioral characteristics — must capture and document explicit user consent before collection begins. Consent records must include the individual's identifier, timestamp, the specific biometric data type, and the purpose of collection.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak biometric data collection consent controls introduces risks including:
Flag each call site not immediately preceded in the same function by a consent retrieval or check (consent, has_consent, or equivalent).
Flag each consent record creation missing any of: individual identifier, ISO 8601 timestamp, biometric data type, collection purpose.
Flag each call site where consent is read from memory or session only and not written to a persistent store before collection.
Flag the file if no revocation or withdrawal function exists anywhere (revoke_consent, withdraw_consent, delete_consent, or equivalent). Count one violation per file, not per call site.
Flag each call site where a consent check exists but the false, null, expired, or error branch does not halt execution before collection.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: None
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: None
Framework
EU AI Act
ISO/IEC 42001
Maryland HB 1202
References
https://mgaleg.maryland.gov/
Biometric data stores must declare a retention limit and deletion scheduling mechanism
AI_DAT_SEC_037
Violation Summary
AI systems that store biometric data must define and enforce a maximum retention period (TTL) for all biometric records and implement automated deletion scheduling to purge biometric data once the retention limit expires. Indefinite retention of biometric data increases exposure to breaches and violates data minimization requirements.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak biometric data retention limit and deletion scheduling controls introduces risks including:
Flag each schema, ORM class, or storage config defining a biometric table or collection that has no TTL field, expires_at column, retention_days constant, or equivalent retention limit.
Flag each biometric data write or insert call targeting a schema that already declares a retention limit but does not set expires_at, TTL, or a retention deadline on the individual record. Do not flag.
Flag the file if no function, scheduled job, cron task, background worker, or database TTL auto-expiry exists to purge biometric records past their retention limit. A function named delete_expired_biometric_records, purge_biometric_data, or equivalent qualifies. Count one violation per file.
Flag each biometric deletion function or job whose body does not contain log statements recording all three of: number of records deleted, deletion timestamp, and retention policy applied. Fields may appear in separate log statements within the same function body.
Flag each code path that modifies expires_at or a retention limit on an existing biometric record without both an explicit administrative override flag and an audit log entry in the same code block.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: None
Framework
EU AI Act
ISO/IEC 42001
Maryland HB 1202
References
https://mgaleg.maryland.gov/
AI consequential decision records must be retained for a minimum of three years
AI_DAT_SEC_038
Violation Summary
Log retention periods, TTL configurations, and storage lifecycle rules for records generated by AI systems involved in consequential decisions must be set to no less than three years (1095 days). This includes decision logs, audit trails, model version identifiers, changelogs, and material update notices. Retention periods below this threshold prevent organizations from demonstrating compliance during regulatory investigations and violate record-keeping obligations under automated decision-making laws.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak ai consequential decision record retention controls introduces risks including:
Constant, variable, or literal value that defines a retention period, TTL, expiry, or log rotation interval for AI decision records and is set to a value less than 1095 days (or its equivalent in seconds, hours, or months: less than 94,608,000 seconds, 26,280 hours, or 36 months).
YAML, JSON, TOML, or .env configuration files that define a log retention, storage lifecycle, or record expiry setting for AI decision data with a value below the three-year minimum.
Terraform, CloudFormation, Kubernetes, or equivalent IaC files that define S3 lifecycle rules, CloudWatch log group retention, database partition expiry, or equivalent storage TTL settings for AI decision records with a retention period below 1095 days.
Logging configuration or logger initialization code that sets a max_age, retention_days, rotate_after, or equivalent parameter to a value below 1095 days for logs that include AI decision records, audit trails, or model version data.
AI decision logging or record storage setup code that does not set any retention period, which defaults to indefinite deletion or storage-provider defaults that may fall below the minimum requirement.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
CERT-In AI Blueprint 2026
California SB 942
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
NIST AI RMF
References
https://leg.colorado.gov/bills/sb26-189
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
AI data stores must enforce encryption at rest and TLS in transit
AI_DAT_SEC_039
Violation Summary
Vector databases, embedding stores, RAG document indexes, training and fine-tuning datasets, prompt and completion logs, and feature stores used by AI systems contain sensitive enterprise data, PII, secrets, and proprietary model context. Storing this data unencrypted or transporting it over plaintext connections exposes it to disk theft, snapshot exfiltration, network interception, and unauthorized internal access. Code that initializes, connects to, or writes to AI data stores must explicitly enable encryption at rest (KMS-managed or AES-256) and require TLS (HTTPS, rediss://, mongodb+srv with tls=true, or equivalent) for every transport.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak encryption at rest and tls in transit controls for ai data stores introduces risks including:
Client initialization or connection strings for vector, embedding, RAG, or AI metadata stores (Pinecone, Weaviate, Qdrant, Milvus, Chroma, pgvector, OpenSearch, Elasticsearch, Redis, MongoDB, PostgreSQL) that use http://, redis://, or mongodb:// without tls=true, or that explicitly set ssl=False, verify=False, use_ssl=False, tls=False, or InsecureSkipVerify=true.
Terraform, CloudFormation, Pulumi, or Kubernetes resources provisioning an AI data store (S3 for training data or embeddings, RDS, DynamoDB, EBS, EFS, Azure Storage, GCS, OpenSearch, ElastiCache, MongoDB Atlas) that omit server_side_encryption_configuration, encryption_at_rest, kms_key_id, or encrypted=true.
Code that writes embeddings, prompts, completions, training data, fine-tuning datasets, or RAG documents via open(), pickle.dump, torch.save, numpy.save, json.dump, or equivalent without wrapping the payload in an authenticated encryption primitive such as Fernet, AES-GCM, KMS Encrypt, or age.
HTTP or gRPC servers (Flask, FastAPI, Express, gRPC) exposing AI model, embedding, RAG, or training endpoints bound to plaintext http or h2c, or AI provider client calls that override the base URL to an http:// scheme.
AI data store credentials, KMS keys, or encryption keys committed in plaintext .env, JSON, or YAML files alongside source code.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EO 14110
EU AI Act
IRAP
ISO/IEC 42001
NIST AI RMF
OWASP-LLM-Top10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/
https://www.nist.gov/itl/ai-rmf
https://www.cert-in.org.in/
Identity and Access Control
Enforces authentication and least-privilege controls between AI agents, MCP servers, and endpoints.
MCP client must authenticate the MCP server
AI_IAC_002
Violation Summary If an MCP client does not authenticate the MCP server, it cannot verify the identity, legitimacy, or trustworthiness of the system providing tool responses, commands, or data. This creates an opportunity for attackers to impersonate the MCP server, intercept or modify traffic, inject malicious tool responses, or deliver falsified data that influences downstream agent reasoning. Without authentication, the MCP trust boundary collapses, enabling man-in-the-middle attacks, data manipulation, workflow corruption, and full compromise of AI-driven operations.
Affected Assets
MCP Server
MCP Client
AI Agent
Severity
Critical
Technical Details Not authenticating the MCP server introduces several risks including:
Server impersonation leading to injection of malicious or misleading responses
Man-in-the-middle interception and modification of MCP traffic
Unauthorized access to sensitive MCP capabilities, tools, and agent operations
Corruption of workflows through falsified output or manipulated data
Leakage of PII or sensitive context exchanged with the illegitimate server
Loss of integrity, trust, and accountability in MCP-driven decisions
Violations of transparency, security, and traceability requirements
Attack Vector: Network/API communication between MCP client and server
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
MCP server must authenticate all clients
AI_IAC_006
Violation Summary If an MCP server does not authenticate the client making requests, any unauthorized entity—including compromised agents, external attackers, or untrusted processes—can impersonate a legitimate client. This allows attackers to invoke privileged tools, access sensitive data, manipulate workflows, or trigger system actions without detection. Lack of client authentication effectively removes all trust boundaries, enabling full compromise of server-side capabilities and AI-driven operations.
Affected Assets
LLM
AI Agent
Severity
Critical
Technical Details Not authenticating the MCP client introduces several risks including:
Unauthorized invocation of high-privilege tools or system actions
Full impersonation of trusted agents, enabling malicious or deceptive requests
Data exposure through unrestricted access to server responses, APIs, or internal systems
Manipulation of downstream workflows via falsified or maliciously crafted requests
Escalation of privilege or lateral movement across connected systems
Loss of accountability, traceability, and auditability for all client-driven actions
Violations of integrity, transparency, and security obligations for regulated AI systems
Attack Vector: Client → MCP server request path
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Inter-agent communication must be authenticated
AI_IAC_007
Violation Summary If agents communicate without authentication, any unauthorized party—including rogue agents, compromised services, or external attackers—can impersonate a legitimate agent and issue commands, request data, or alter system behavior. Non-authenticated inter-agent communication destroys trust boundaries between autonomous components and enables impersonation, privilege escalation, data leakage, workflow manipulation, and full compromise of multi-agent systems. Without identity guarantees, agent-to-agent messaging becomes a high-risk attack surface.
Affected Assets
AI Agent
Severity
Critical
Technical Details Missing authentication between agents introduces several risks including:
Unauthorized agents impersonating trusted components to issue commands
Manipulation of agent workflows or decision chains through falsified messages
Leakage of sensitive data exchanged during inter-agent coordination
Injection of malicious instructions into distributed reasoning processes
Loss of accountability and inability to attribute harmful actions
Increased lateral movement risk across agent networks
Violations of integrity, trust, and regulatory controls for autonomous systems
Attack Vector: Inter-agent message channel / network communication
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: Critical
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-ASI-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
Agents must not hold excessive external system credentials
AI_IAC_008
Violation Summary When agents are configured with credentials to access more than three external systems, the blast radius of a compromise dramatically increases. Each additional credential expands the agent’s privilege footprint and creates new pathways for lateral movement, data exfiltration, unauthorized actions, and multi-system compromise. Over-privileged agents become single points of systemic failure if the agent is hijacked, attacked, misconfigured, or manipulated by an LLM prompt, all connected external systems are at risk simultaneously.
Affected Assets
AI Agent
Severity
High/Critical
Technical Details Allowing an agent to hold multiple (3+) external system credentials introduces several risks including:
Large blast radius: compromising the agent compromises all connected systems
Increased likelihood of credential leakage through logs, LLM outputs, prompts, or tool interactions
Prompt injection enabling unauthorized use of high-privilege multi-system access
Lateral movement across different platforms (e.g., Jira → GitHub → AWS → Snowflake)
Violation of least-privilege and separation-of-duties principles
Difficulty revoking or rotating credentials in incident response
Loss of governance and traceability when many systems are accessed through one agent identity
Attack Vector: Agent credential store / agent-initiated external API calls
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: Critical
Availability Impact: Medium to High
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-ASI-Top10
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://artificialintelligenceact.eu/ai-act-explorer/
A user must authenticate before accessing the AI Agent
AI_IAC_014
Violation Summary
Allowing access to an AI Agent without user authentication enables unauthorized actors to invoke agent capabilities, consume resources, and perform malicious actions under anonymous or unverified identities. This weakens accountability, bypasses access governance, and increases the risk of abuse in production and enterprise environments.
Affected Assets
AI Agent
Severity
High
Technical Details
Missing authentication before AI Agent access introduces several risks including:
Unauthorized use of AI Agent functionality by external or internal actors
Bypass of identity-based controls and audit accountability
Potential misuse for malicious prompting, automation abuse, or harmful output generation
Increased exposure of sensitive responses or restricted agent behaviors to unverified users compliance and governance violations related to access control requirements
Attack Vector: Network/Application Access
Attack Complexity: Low
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-ASI-Top10
References https://genai.owasp.org/initiatives/agentic-security-initiative/ https://genai.owasp.org/llm-top-10/ https://artificialintelligenceact.eu/ai-act-explorer/
Enforce URL allowlists for agent fetches, tools, and outbound HTTP
AI_IAC_015
Violation Summary
Unrestricted URLs let prompt injection or tool abuse drive server-side request forgery, data exfiltration, and lateral movement. LLM and agentic security guidance and operational controls for AI systems expect explicit allowlisting and scheme restrictions for any automated retrieval or callback.
Affected Assets
AI Model
MCP Server
LLM
Severity
Critical
Technical Details
Missing or weak URL allowlists for agent fetches, tools, and outbound http controls introduces risks including:
Allowing arbitrary URL fetches enables attackers to use the AI agent to execute server-side request forgery (SSRF), accessing sensitive internal networks, link-local addresses, and cloud metadata endpoints.
Permitting unvalidated model-chosen or user-supplied URLs without strict host and path rules allows malicious actors to covertly exfiltrate data to external attacker-controlled servers.
Permitting dangerous or custom schemes (such as file://, gopher://, dict://, or ftp://) instead of enforcing a default deny-list allows attackers to pivot and move laterally within the internal network.
Failing to re-validate each HTTP redirect hop against the allowlist allows attackers to bypass initial URL checks and redirect the agent to restricted targets.
Attack Vector: Network
Attack Complexity: Low
Privileges Required: None to Low
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low to Medium
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Detect and block agent privilege escalation attempts
AI_IAC_016
Violation Summary
When agent orchestration does not detect and block privilege escalation, attackers or flawed automation can expand OAuth scopes, administrative APIs, roles, or dangerous tools beyond what the user or policy approved. That breaks least-privilege expectations for agentic systems and enables unauthorized or destructive actions with the appearance of legitimate agent behavior.
Affected Assets
AI Agent
LLM
MCP Server
Severity
Critical
Technical Details
Missing or weak escalation controls introduces risks including:
Unbounded scope and capability growth driven by model output, chat, or tool results without human-in-the-loop approval or static policy comparison
Dynamic registration or use of admin-only, destructive, or credential-bearing tools without verifying the bound identity’s role
Role or permission changes triggered by model-generated content instead of governed identity workflows
Execution of tool plans without comparing requested tools, HTTP methods, or resource paths to a maximum-privilege profile captured at bind time
Fail-open behavior that logs suspected escalation but still executes the action
Attack Vector: Network / Application (abuse of agent tooling and authorization flows)
Attack Complexity: Low to Medium
Privileges Required: Low (often none beyond normal agent or user session use)
User Interaction: None to Limited
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low to Medium
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-ASI-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Maintain session token integrity with signing, verification, expiry, and binding
AI_IAC_017
Violation Summary
Weak or improperly handled session tokens for agents, APIs, or MCP clients enable forgery, session fixation, confused-deputy behavior, and long-lived unauthorized access. Without integrity, authenticity, expiry, binding, and consistent validation, session identifiers become a high-value target and a weak control for delegated agent actions.
Affected Assets
AI Agent
LLM
MCP Server
MCP Client
Severity
Critical
Technical Details
Inadequate session token design and validation introduce risks including:
Accepting unsigned or unauthenticated tokens (e.g. trusting opaque random IDs with no MAC or signature verification)
Guessable or sequential identifiers instead of cryptographically secure randomness
Missing or unenforced expiry and lack of server-side invalidation
Timing-sensitive comparison of secrets (e.g. plain == on HMACs or bearer tokens) exposing tokens to guessing or side channels
Session tokens in URLs, logs, or client-visible errors (leakage and replay)
Continuing to honor old session tokens after role or scope elevation without re-issuing bound tokens
Attack Vector: Network
Attack Complexity: Low to Medium
Privileges Required: None to Low
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-ASI-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Enforce cryptographically verified user-to-agent binding for every request
AI_IAC_018
Violation Summary
Without binding, attackers or confused deputies can steer an agent with another user's identity, reuse pooled agents across principals, or override the authenticated subject using client-supplied fields or model text. AI governance expectations and agentic identity guidance require stable, verified binding between human or service identity and agent execution context.
Affected Assets
AI Agent
LLM
MCP Server
Severity
Critical
Technical Details
Missing or weak cryptographically verified user-to-agent binding for every request control introduces risks including:
Threat actors overriding authenticated subjects by supplying unverified JSON fields, query parameters, or unsigned headers, allowing them to impersonate other users for tool calls, MCP invocations, and downstream API requests.
Parsing LLM or tool output to change the bound user, role, tenant, or OAuth subject allows the model or attacker to manipulate the execution identity before privileged actions.
Reusing a single long-lived agent instance, connection, or MCP session across different HTTP requests or WebSocket sessions without explicitly re-initializing and re-verifying identity allows attackers to hijack pooled agents and access data across principals.
Invoking high-risk tools (e.g., payments, admin APIs, PII export) without explicitly verifying that the currently bound subject matches the resource owner or an authorized delegate enables unauthorized actions.
Continuing agent execution with a default or anonymous user when authentication or binding validation fails provides a fallback mechanism that attackers can exploit to bypass access controls entirely.
Attack Vector: Network
Attack Complexity: Low
Privileges Required: None to Low
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Low to Medium
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Restrict AI agents to an explicit tool allowlist
AI_IAC_SEC_020
Violation Summary
AI agents that can invoke arbitrary local or remote tools are highly susceptible to prompt-injection-driven abuse, privilege creep, and unintended data exfiltration. Enforcing a centrally managed allow list ensures only approved capabilities are reachable and prevents fallback to broad or dynamic tool access.
Affected Assets
AI Agent
LLM
MCP Server
MCP Client
Severity
Critical
Technical Details
Missing or weak restrict ai agents to an explicit tool allow list controls introduces risks including:
Executing tools from a registry, plugin catalog, MCP server list, or runtime-discovered manifest without verifying membership in an approved allow list.
Permitting configurations that use '*' tool names, default-all behavior, or interpret empty allow lists as granting full access.
Enabling or adding tools directly from LLM outputs, prompt text, or untrusted metadata without server-side policy validation.
Reusing a single global allow list across distinct user roles or task classes instead of applying narrower, scoped lists.
Proceeding with tool execution when an allow list fetch, parse, signature check, or policy service lookup fails, rather than failing closed.
Failing to log attempted tool IDs, actors, policy versions, and denial reasons to a protected audit sink when unauthorized tools are blocked.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: None to Low
User Interaction: None to Limited
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
EU AI Act
IRAP
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Enforce resource bounds, termination limits, and traceability for subagent spawning
AI_IAC_022
Violation Summary
Unbounded spawning of AI subagents without strict lifecycle constraints can lead to runaway autonomous processes, infinite loops, and 'agentic fork-bombs' that exhaust compute, memory, or API budgets. Agent orchestration code must enforce hard timeouts, maximum execution step counts, concurrency limits, and structured parent-child traceability at every point a subagent is created.
Affected Assets
AI Agent
LLM
Severity
High
Technical Details
Missing or weak autonomous subagent spawning and execution controls introduces risks including:
Unsafe or inconsistent agent behavior from missing execution bounds (no timeouts or step counts), allowing subagents to run indefinitely and exhaust compute or API quotas.
Agentic fork-bomb conditions arising from loop-based or recursive spawning with no counter guard, causing uncontrolled replication of child processes.
Prompt/task injection via unvalidated instruction passthrough, raw LLM or upstream agent outputs passed directly to subagents without format verification or assertion checks.
Loss of forensic traceability when spawn events lack correlation or parent-agent identifiers, making incident reconstruction and chain-of-custody auditing impossible.
Resource exhaustion (memory, API budget, concurrency slots) due to absent concurrency limits and unbounded spawning across parallel workflows.
Compliance and governance failures stemming from absent structured audit logs at spawn sites, preventing detection of policy violations and accountability attributes.
Attack Vector: Network
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: Low
Integrity Impact: High
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/initiatives/agentic-security-initiative/
Chatbot and AI interfaces must disclose AI identity to the user
AI_IAC_023
Violation Summary
AI systems that interact directly with natural persons must inform users they are communicating with an AI. When chatbot and conversational AI interfaces fail to disclose their AI nature, users are deceived into believing they are engaging with a human agent. This undermines informed consent, enables psychological manipulation, and violates transparency obligations under the EU AI Act Article 50. Suppressing or omitting AI identity disclosure exposes organizations to regulatory penalties and erodes user trust.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
Critical
Technical Details
Failing to disclose AI identity in chatbot and conversational interfaces introduces several risks including:
Users deceived into believing they are communicating with a human, enabling manipulation and false trust
Inability of users to make informed decisions about disclosing private or sensitive details
Regulatory penalties and legal liability arising from non-compliance with mandatory disclosure requirements
Attack Vector: Network
Attack Complexity: Low
Privileges Required: None
User Interaction: Required
Confidentiality Impact: Medium
Integrity Impact: Medium
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
California SB 942
China CAC GenAI Measures
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
South Korea AI Framework Act
Vietnam AI Law
References
https://artificialintelligenceact.eu/ai-act-explorer/
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
General purpose AI model integrations must reference a model card or technical documentation
AI_IAC_024
Violation Summary
Code that integrates a general-purpose AI (GPAI) model such as GPT, Claude, Gemini, Mistral, or Llama must reference a model card or technical documentation in the same module. When model card references are absent, developers, auditors, and incident responders cannot verify model provenance, capabilities, known limitations, or bias characteristics. This absence creates compliance gaps under EU AI Act Article 53, undermines AI system auditability, and increases the risk of deploying models with undisclosed failure modes or unsafe behaviors.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
Medium
Technical Details
Failing to reference a model card or technical documentation in GPAI integrations introduces several risks including:
Inability to audit AI model provenance, versioning, and known limitations during development and review
Deployment of deprecated, unvalidated, or unsafe model versions without awareness of documented risks
Developers and auditors unable to assess model risk, biases, or behavioral constraints without documentation reference
Lack of traceability for AI-generated outputs during incident response and forensic investigation
Hidden use of models with known biases, harmful capabilities, or usage restrictions not communicated to the team
Attack Vector: Local
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: Low
Integrity Impact: Medium
Availability Impact: Low
Framework
California-2023-AB2013-Chaptered
China CAC GenAI Measures
EU AI Act
NIST AI RMF
OWASP-LLM-Top10
References
https://artificialintelligenceact.eu/ai-act-explorer/
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Healthcare AI systems must disclose AI use to patients before diagnosis, treatment, or imaging
AI_IAC_025
Violation Summary
AI systems used in healthcare settings for diagnosis, treatment recommendations, or medical imaging analysis must explicitly disclose their AI nature to patients prior to generating clinical output. Code must include pre-invocation disclosure checks and patient-facing responses must carry disclosure notices in compliance with California AB 3030.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
High
Technical Details
Missing or weak healthcare ai disclosure controls before clinical procedures introduces risks including:
Invokes an AI model for clinical purposes (diagnosis, treatment recommendation, imaging analysis, or clinical decision support) without a preceding check that a patient disclosure notice has been presented and confirmed.
LLM system prompt string used in a healthcare or clinical context that does not include a statement identifying the assistant as an AI and disclosing its use in clinical support.
Formats or sends a clinical AI response to a patient without prepending or appending a disclosure string indicating the content was generated by an AI system.
Or prompt instruction that omits, bypasses, or instructs the model to conceal its AI nature in a clinical context.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: Required
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
California AB3030 Healthcare AI
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
References
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
https://aibasicact.kr/
AI clinical recommendations must disclose that human clinicians retain final decision authority
AI_IAC_026
Violation Summary
AI systems that generate clinical recommendations, diagnostic suggestions, or treatment plans must include a disclosure that human clinicians remain responsible for all final medical decisions. AI output must not be presented as definitive clinical judgement without a human oversight disclaimer.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
High
Technical Details
Missing or weak ai clinical recommendation disclosure of human clinician decision authority introduces risks including:
Formats or returns a clinical AI response (diagnosis, treatment recommendation, imaging finding, or clinical decision support output) whose returned value does not include or reference a human-clinician final-authority disclaimer; a response that concatenates or references such a disclaimer is compliant.
LLM system prompt used in a clinical context that does not state that AI output is advisory only and that licensed clinicians are responsible for final decisions.
Or prompt string that explicitly asserts AI-generated clinical output is a definitive or final medical decision without a human review or oversight qualifier in the same string.
Ignores or rejects a clinician-supplied override of an AI-generated clinical recommendation; code that honors a clinician override when provided is compliant.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: Required
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
California AB3030 Healthcare AI
ISO/IEC 42001
Japan AI Basic Plan
NIST AI RMF
South Korea AI Framework Act
References
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
https://aibasicact.kr/
https://www.cas.go.jp/
Healthcare AI communications must include patient instructions to request human-only review
AI_IAC_027
Violation Summary
AI systems communicating clinical information to patients must include clear instructions enabling patients to request a human-only review of any AI-generated recommendation. Patient-facing clinical AI responses must provide a direct path to reach a human healthcare provider in compliance with California AB 3030.
Affected Assets
AI Agent
LLM Application
AI Model
Severity
Medium
Technical Details
Missing or weak healthcare ai patient instructions for requesting human-only review introduces risks including:
Delivers a clinical AI response to a patient without including instructions or a mechanism for the patient to request a human-only review of the AI-generated recommendation.
Patient-facing clinical AI response handler that does not include or reference contact information, escalation path, or opt-out mechanism enabling the patient to reach a human healthcare provider.
LLM system prompt used in a patient-facing clinical context that does not instruct the model to inform patients how to contact a human provider or request human review.
Or prompt instruction that blocks, removes, or suppresses the patient's ability to request human review or contact a human provider.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: Required
Confidentiality Impact: None
Integrity Impact: Medium
Availability Impact: None
Framework
California AB3030 Healthcare AI
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
References
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
https://aibasicact.kr/
AI system deployments must declare a risk classification level in configuration metadata
AI_IAC_028
Violation Summary
AI system deployment code and configuration must include a declared risk classification level (low, medium, or high) that enables risk-proportionate governance, oversight, and compliance obligations. Without an explicit risk classification, organizations cannot apply tiered controls or demonstrate compliance with risk-based regulatory frameworks.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak ai system deployment risk classification declaration controls introduces risks including:
AI deployment configuration files (YAML, JSON, TOML, .env, or equivalent) that define an AI model, agent, or service deployment without a risk_level, risk_classification, or risk_category field set to one of: low, medium, or high.
AI model serving entrypoints, agent initialization, or application startup code that instantiates an AI system without loading, referencing, or validating a risk classification value from configuration or metadata.
Configuration files or code that set the risk classification to a value other than low, medium, or high (or equivalent enumerated values defined by the system).
AI system startup code that loads a risk classification value without validating it against the allowed set of values before proceeding with initialization.
AI systems where the risk classification is declared in configuration but not passed to logging, monitoring, access control, or governance modules that need it to apply risk-proportionate controls.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
EU AI Act
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
Vietnam AI Law
References
https://chinhphu.vn/
https://www.data.gov.au/
https://www.digitaltransformationoffice.gov.au/
https://www.nist.gov/itl/ai-rmf
https://aibasicact.kr/
AI model deployments must maintain version tracking, change logs, and release documentation
AI_IAC_029
Violation Summary
AI model deployment code and configuration must implement version tracking that uniquely identifies each deployed model version, maintain change logs documenting modifications between versions, and provide release documentation covering model capabilities, limitations, and deployment parameters. Without version tracking and change documentation, organizations cannot audit model behavior, reproduce results, or demonstrate regulatory compliance.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
Medium
Technical Details
Missing or weak ai model deployment version tracking and documentation controls introduces risks including:
AI model loading, serving, or deployment code that instantiates or references a model without a version identifier (semantic version, hash, date stamp, or unique release tag) in the configuration, model path, or metadata.
AI inference endpoints or response handlers that return predictions or recommendations without including the model version identifier in the response payload, headers, or metadata.
AI model deployment repositories or configuration directories that lack a change log file (CHANGELOG, CHANGES, release notes, or equivalent) documenting what changed between model versions.
Model deployment configurations that lack a model card, release manifest, or documentation file specifying model capabilities, known limitations, training data summary, and evaluation metrics.
AI inference logging code that writes structured log entries without including the model version identifier, making post-incident analysis and audit trail reconstruction infeasible.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: Medium
Availability Impact: None
Framework
AU Gov AI Policy v2.0
AU National Framework
California SB 942
California-2023-AB2013-Chaptered
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
NIST AI RMF
Vietnam AI Law
References
https://leginfo.legislature.ca.gov/
https://www.nist.gov/itl/ai-rmf
https://www.data.gov.au/
https://leg.colorado.gov/bills/sb26-189
https://chinhphu.vn/
https://www.digitaltransformationoffice.gov.au/
AI system deployments must declare a covered domain classification per automated decision-making regulations
AI_IAC_030
Violation Summary
Every AI system deployment configuration or manifest must include a covered_domain field that identifies which regulated domain the system operates in, using the taxonomy defined by automated decision-making regulations: education, employment, real_estate, financial_services, insurance, healthcare, government_benefits, or not_applicable. This declaration is the prerequisite trigger for all downstream compliance obligations including consumer disclosure, human review, record retention, and adverse outcome notification. Without an explicit covered domain declaration, organizations cannot demonstrate regulatory scoping decisions and cannot apply or waive the correct compliance controls.
Affected Assets
AI Model
AI Agent
LLM Application
Severity
High
Technical Details
Missing or weak ai system deployment covered domain classification controls introduces risks including:
AI deployment configuration files (YAML, JSON, TOML, .env, or equivalent) that define an AI model, agent, or application deployment without a covered_domain or equivalent field.
Configuration files or code that set covered_domain to a value outside the allowed taxonomy: education, employment, real_estate, financial_services, insurance, healthcare, government_benefits, not_applicable. Flag catch-all values such as 'other', 'unknown', 'general', or empty strings.
AI system initialization or startup code that reads a deployment configuration without validating that the covered_domain field is present and set to a value from the allowed taxonomy before the system begins processing requests.
Configurations that set covered_domain to not_applicable without an accompanying justification comment or field explaining why the system does not operate in any regulated domain.
AI systems where the covered_domain is declared in configuration but is not passed to disclosure, logging, retention, or human review modules that require it to apply or waive domain-specific compliance obligations.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: None
Framework
Colorado SB 26-189
EU AI Act
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
References
https://leg.colorado.gov/bills/sb26-189
https://aibasicact.kr/
Vulnerability
Continuously detects and remediates critical software weaknesses inside AI assets.
Do not allow dependencies with critical or high severity vulnerabilities
AI_VULN_SEC_001
Violation Summary Allowing AI agents to depend on libraries or packages with known critical or high-severity vulnerabilities introduces a direct and exploitable attack surface into the agent runtime. AI agents typically operate with elevated privileges, access sensitive data, invoke tools, and interact with external systems. Vulnerable dependencies can be exploited to execute arbitrary code, escalate privileges, leak secrets, poison agent behavior, or compromise downstream systems—often without direct interaction with the LLM itself.
Affected Assets
AI Agent
MCP Server
LLM
Severity
Critical
Technical Details
Using dependencies with critical or high vulnerabilities introduces several risks including:
Remote code execution or arbitrary command execution within the agent environment
Credential theft, token leakage, or exposure of secrets used by the agent
Supply-chain attacks where malicious code is introduced via compromised packages
Manipulation or poisoning of agent logic, tool invocation, or decision flows
Lateral movement across systems accessed by the agent
Persistence mechanisms established through compromised libraries
Inability to trust agent outputs or actions due to compromised runtime integrity
Violations of secure development, patch management, and risk-management obligations
Attack Vector: Vulnerable third-party dependency / supply-chain compromise
Attack Complexity: Low
Privileges Required: None (exploits often execute with agent privileges)
User Interaction: None
Confidentiality Impact: Critical
Integrity Impact: Critical
Availability Impact: Medium to High
Framework
OWASP LLM: LLM05, LLM08
OWASP ASI: ASI-05
NIST SSDF: Practice RV.1
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://csrc.nist.gov/pubs/sp/800/218/final
Do not allow critical or high vulnerabilities in the code
AI_VULN_SEC_002
Violation Summary The presence of vulnerabilities in application or agent code introduces direct security risks that can be exploited to compromise confidentiality, integrity, and availability. Vulnerable code paths—such as injection flaws, insecure deserialization, broken authentication, improper authorization, or unsafe file handling—can be abused by attackers to execute arbitrary code, access sensitive data, manipulate AI behavior, or disrupt operations. In AI and agent-based systems, these vulnerabilities are especially dangerous because compromised code can influence autonomous decisions and propagate impact across multiple systems.
Affected Assets
AI Agent
MCP Server
LLM
Severity
Critical
Technical Details
Having vulnerabilities in code introduces several risks including:
Remote code execution or command injection through exploitable code paths
Unauthorized access to sensitive data, credentials, or internal APIs
Manipulation of AI agent logic, reasoning flows, or tool invocation
Privilege escalation or bypass of authorization controls
Lateral movement across integrated systems and services
Persistence mechanisms established through exploited vulnerabilities
Loss of trust in application outputs and automated decisions
Violations of secure development lifecycle and regulatory requirements
Attack Vector : Vulnerable application or agent code
Attack Complexity: Low to Medium (depending on vulnerability type)
Privileges Required: None (for many common vulnerabilities)
User Interaction: None or minimal
Confidentiality Impact: Critical
Integrity Impact: Critical
Availability Impact: Medium to High
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
EO 14110
EU AI Act
IRAP
NIST AI RMF
NIST SSDF
OWASP-Top-10
Vietnam AI Law
References
https://genai.owasp.org/llm-top-10/ https://genai.owasp.org/initiatives/agentic-security-initiative/ https://csrc.nist.gov/pubs/sp/800/218/final
Enforce foundation model identity, version pinning, and approved model registry for all AI workloads
AI_VULN_SEC_005
Violation Summary
Using unverified, unversioned, or unregistered foundation models exposes the system to supply-chain compromise, silent model swaps, and untracked behavioral drift. AI risk management third-party supply chain controls and pre-trained model monitoring expectations require every model loaded at inference or fine-tuning time to be resolved from an approved registry with cryptographic identity verification and pinned versions.
Affected Assets
AI Model
AI Agent
MCP Server
LLM
Severity
Critical
Technical Details
Missing foundation model identity, version pinning, and approved model registry for all AI workloads controls introduces risks including:
Loading models from arbitrary, unregistered URLs or paths exposes the system to malicious or tampered artifacts and supply-chain compromise.
Allowing user input or prompt text to select active models or failing to verify cryptographic hashes/signatures against a known-good manifest, permits unauthorized execution and silent model swaps.
Using mutable tags (like `latest`) instead of immutable identifiers or digest pins causes untracked behavioral drift, making it impossible to guarantee consistent model behavior over time.
Serving predictions without recording the resolved model identifier and version in request metadata prevents effective forensic readiness, monitoring, and accountability.
Failing open and proceeding to load models when registry lookups or hash verifications fail bypasses critical security gates and leads to insecure fallback states.
Attack Vector: Network
Attack Complexity: Medium to High
Privileges Required: High
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
California-2023-AB2013-Chaptered
Colorado SB 26-189
EO 14110
EU AI Act
NIST AI RMF
OWASP-ASI-Top10
OWASP-LLM-Top10
References
https://genai.owasp.org/llm-top-10/
https://genai.owasp.org/initiatives/agentic-security-initiative/
Memory safety and buffer overflow prevention in native AI code (C/C++/Rust)
AI_VULN_SEC_006
Violation Summary
Native C, C++, and Rust used in AI workloads can suffer memory corruption and undefined behavior when bounds, ownership, and concurrency are not enforced, enabling crashes, data leakage, and remote code execution along the AI inference or tooling path.
Affected Assets
LLM
MCP Server
AI Agent
Severity
Critical
Technical Details
Unsafe native patterns in AI-related code introduce risks including:
Unbounded or unchecked copies (strcpy, strcat, gets, sprintf, unsafe memcpy/memmove/memset lengths)
Off-by-one and wrapping size arithmetic that exceeds allocated objects
Stack buffers filled from untrusted network, file, IPC, or tensor dimensions without clamping
Double free, use-after-free, invalid free, and ambiguous ownership of raw pointers
Missing paired release on all return paths (including exceptions) without RAII
Unsound Rust unsafe (raw deref from unvalidated input, bad transmute, unchecked indexing)
Data races on shared buffers without locks or atomics
Disabled sanitizers or bounds checks in production builds
Attack Vector: Network
Attack Complexity: High
Privileges Required: None
User Interaction: None
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: High
Framework
CERT-In AI Blueprint 2026
IRAP
NIST AI RMF
OWASP-LLM-Top10
References
AI systems must implement incident detection, structured logging, and reporting mechanisms
AI_VULN_SEC_007
Violation Summary
AI systems must maintain structured logging for all inference, decision, and agent operations, implement anomaly and incident detection, and provide reporting mechanisms that enable timely notification to authorities and stakeholders when incidents occur. Without these controls, AI failures, misuse, and harm go undetected and unreported.
Affected Assets
AI Model
AI Agent
LLM Application
MCP Server
Severity
High
Technical Details
Missing or weak ai system incident detection, structured logging, and reporting mechanism controls introduces risks including:
AI inference endpoints, agent execution paths, or decision pipelines that do not write structured log entries (JSON or key-value format) capturing at minimum: timestamp, request identifier, model identifier, input summary, output summary, and outcome status.
AI system code that lacks error rate monitoring, anomaly detection hooks, threshold-based alerting, or health-check endpoints that can surface degraded performance, unexpected outputs, or elevated failure rates.
AI system deployments that lack a reporting function, webhook, or integration to notify operators, administrators, or regulatory contacts when a detected incident exceeds severity thresholds.
AI inference or agent error handlers that catch exceptions without writing a structured log entry containing the error type, stack context, input that triggered the failure, and a severity classification.
AI logging configurations that lack a retention policy, log rotation, or export path to a durable store, making post-incident investigation infeasible.
Attack Vector: Network / Application
Attack Complexity: Low
Privileges Required: Low
User Interaction: None
Confidentiality Impact: None
Integrity Impact: High
Availability Impact: Low
Framework
AU Gov AI Policy v2.0
AU National Framework
CERT-In AI Blueprint 2026
China CAC GenAI Measures
EU AI Act
IRAP
ISO/IEC 42001
NIST AI RMF
South Korea AI Framework Act
Vietnam AI Law
References
https://chinhphu.vn/
https://www.airuniversity.af.edu/CASI/
https://www.digitaltransformationoffice.gov.au/
https://www.data.gov.au/
https://www.cyber.gov.au/
https://www.nist.gov/itl/ai-rmf
https://aibasicact.kr/
Skills
Automatically discovers, scans, and tracks skills in your inventory.
Do not allow malicious skills
AI_SKILL_SEC_001
Violation Summary
Use of malicious or untrusted skills introduces risks of unauthorized actions, data theft, remote command execution, privilege abuse, and compromise of AI agent integrity.
Affected Assets
LLM
AI Agent
Skill
Severity
Critical
Technical Details
Malicious skills introduce several risks including:
Unauthorized execution of privileged actions
Data exfiltration through tool or API access
Remote code execution through unsafe tool invocation
Abuse of implicit trust between agent and skill
Credential theft or token misuse
Manipulation of agent reasoning and decision-making
Persistence through malicious extensions or runtime modifications
Supply chain compromise through third-party skills
Bypass of governance, policy, or approval controls
Unsafe autonomous behavior and unintended tool chaining
Attack Vector: Skill Extension
Attack Complexity: Low
Privileges Required: Low to None
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium
Framework
OWASP-Agentic-Skills-Top10
References
https://genai.owasp.org/llm-top-10
https://genai.owasp.org/initiatives/agentic-security-initiative
https://owasp.org/www-project-agentic-skills-top-10
Do not allow suspicious skills
AI_SKILL_SEC_002
Violation Summary
Use of malicious or untrusted skills introduces risks of unauthorized actions, data theft, remote command execution, privilege abuse, and compromise of AI agent integrity.
Affected Assets
LLM
AI Agent
Skill
Severity
High
Technical Details
Malicious skills introduce several risks including:
Unauthorized execution of privileged actions
Data exfiltration through tool or API access
Remote code execution through unsafe tool invocation
Abuse of implicit trust between agent and skill
Credential theft or token misuse
Manipulation of agent reasoning and decision-making
Persistence through malicious extensions or runtime modifications
Supply chain compromise through third-party skills
Bypass of governance, policy, or approval controls
Unsafe autonomous behavior and unintended tool chaining
Attack Vector: Skill Extension
Attack Complexity: Low
Privileges Required: Low to None
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium
Framework
OWASP-Agentic-Skills-Top10
References
https://genai.owasp.org/llm-top-10
https://genai.owasp.org/initiatives/agentic-security-initiative
https://owasp.org/www-project-agentic-skills-top-10/
Do not allow unknown skills
AI_SKILL_SEC_003
Violation Summary
Use of malicious or untrusted skills introduces risks of unauthorized actions, data theft, remote command execution, privilege abuse, and compromise of AI agent integrity.
Affected Assets
LLM
AI Agent
Skills
Severity
High
Technical Details
Malicious skills introduce several risks including:
Unauthorized execution of privileged actions
Data exfiltration through tool or API access
Remote code execution through unsafe tool invocation
Abuse of implicit trust between agent and skill
Credential theft or token misuse
Manipulation of agent reasoning and decision-making
Persistence through malicious extensions or runtime modifications
Supply chain compromise through third-party skills
Bypass of governance, policy, or approval controls
Unsafe autonomous behavior and unintended tool chaining
Attack Vector: Skill Extension
Attack Complexity: Low
Privileges Required: Low to None
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium
Framework
OWASP-Agentic-Skills-Top10
References
https://genai.owasp.org/llm-top-10
https://genai.owasp.org/initiatives/agentic-security-initiative
https://owasp.org/www-project-agentic-skills-top-10
Do not allow skills that exfiltrate data
AI_SKILL_DAT_SEC_001
Violation Summary
Allowing AI agents to use skills that can exfiltrate data introduces significant confidentiality, integrity, operational, and regulatory risks. Modern agent skills often execute with the same permissions and credentials as the hosting agent, allowing them to access API keys, tokens, SSH credentials, browser data, cloud storage, enterprise documents, memory stores, and internal systems.
Affected Assets
LLM
AI Agent
Skills
Technical Details
Key risks include:
Unauthorized disclosure of sensitive data including PII, credentials, intellectual property, financial records, source code, and regulated data.
Abuse of inherited agent permissions to access systems beyond the user’s intent.
Covert exfiltration through legitimate APIs, retrieval tools, cloud synchronization, or outbound network calls.
Prompt-driven manipulation where instruction files steer agents into leaking secrets.
Regulatory exposure under privacy and AI governance regulations when protected data is leaked.
Loss of trustworthiness and governance due to opaque autonomous behavior and insufficient auditability.
Attack Vector: Skill Extension
Attack Complexity: Low
Privileges Required: Low to None
User Interaction: Required
Confidentiality Impact: High
Integrity Impact: High
Availability Impact: Medium
Framework
OWASP-Agentic-Skills-Top10
References
https://genai.owasp.org/llm-top-10
https://genai.owasp.org/initiatives/agentic-security-initiative
Last updated