AI security is the practice of protecting AI systems, especially large language models, from attacks that manipulate their inputs, leak their data, or hijack their actions. This glossary defines the core terms in plain language.
Clear definitions of the attacks, defenses, and compliance terms that matter when you secure LLMs, chatbots, and AI agents. Terms map to the OWASP LLM Top 10.
Prompt injection is an attack where a user hides instructions inside the text an AI model reads, causing the model to ignore its original rules and follow the attacker instead. It is the most common LLM attack and the root of many data leaks and jailbreaks.
Indirect prompt injection delivers the malicious instructions through content the AI retrieves rather than what the user types, for example a poisoned web page, document, or email that an AI agent reads. The user never sees the attack, but the model obeys the hidden instructions.
A jailbreak is an input crafted to bypass an AI model safety guardrails, making it produce content or take actions it was designed to refuse. Jailbreaks often use role-play, encoding, or step-by-step framing to slip past filters.
Data exfiltration in AI security is any technique that causes a model to reveal information it should keep private, such as its system prompt, internal documents, or the personal data of other users. It is a primary goal of many prompt injection attacks.
System prompt leaking is when an attacker tricks a model into revealing its hidden system prompt, the configuration text that defines its role and rules. A leaked system prompt hands attackers a map for crafting more effective jailbreaks.
Agent hijacking targets AI agents that can use tools, call APIs, or run code. By manipulating the agent inputs, an attacker causes it to take unauthorized actions, such as querying a database, sending a message, or escalating its own privileges.
Model poisoning corrupts an AI model behavior by tampering with its training data or fine-tuning process, planting hidden triggers or biases. The model looks normal until a specific input activates the malicious behavior.
A hallucination is when an AI model generates confident, fluent output that is factually wrong or entirely made up. In a security context, hallucinations become dangerous when a model invents data, citations, or instructions that users or downstream systems trust.
AI red teaming is the practice of attacking your own AI system on purpose, using adversarial prompts across known threat categories, to find weaknesses before real attackers do. The output is a documented set of findings and fixes, the AI equivalent of penetration testing.
An AI firewall is a security layer that sits between users and an AI model in production, inspecting every prompt and response in real time to block prompt injection, jailbreaks, PII leaks, and policy violations before they reach the model or the user.
Guardrails are the rules and filters that constrain what an AI system will accept as input and produce as output. They range from simple keyword blocks to AI-based classifiers, and they are most effective when layered rather than relied on alone.
PII redaction automatically detects and removes personal information, such as names, emails, and account numbers, from AI inputs and outputs. It is a core control for privacy compliance and for preventing a model from leaking sensitive data in its responses.
Adversarial testing systematically probes an AI system with inputs designed to make it fail, measuring how it responds to attacks rather than to normal use. It is the testing method behind AI red teaming and a documented requirement in maturing AI security programs.
The OWASP LLM Top 10 is an industry-standard list of the most critical security risks for large language model applications, including prompt injection, data leakage, and insecure output handling. It is the common reference framework for AI security testing.
RAG is an architecture where an AI model retrieves documents from a knowledge base and uses them to answer a question. It improves accuracy, but it also creates an attack surface, because poisoned retrieved content can carry indirect prompt injection.
Bill C-8 is Canada’s federal cybersecurity law for critical infrastructure, which received Royal Assent in June 2026. It requires designated operators to run cybersecurity programs, report incidents, and manage supply chain risk, and its requirements extend to the AI systems those operators run. See our Bill C-8 guide.
PIPEDA is Canada’s federal private-sector privacy law. For AI systems, it governs how personal information can be collected, used, and protected, which makes PII redaction, data residency, and audit logging important controls for any AI handling Canadian personal data.
ISO/IEC 42001 is the international management-system standard for artificial intelligence. It sets out requirements for AI risk assessment, monitoring, and accountability, and it is the certification Canadian and global buyers increasingly ask AI vendors about.
Data residency is the requirement that data be stored and processed within a specific country or region. For Canadian organizations using AI, Canadian data residency keeps prompts, responses, and logs inside Canada, which simplifies compliance with PIPEDA and Bill C-8.
An audit log is a complete, tamper-evident record of activity. For AI systems, it captures every prompt and response, which is essential evidence for incident reporting, regulatory review, and demonstrating that AI security controls are actually working.
SecuritAI combines AI red teaming and a real-time AI firewall in one platform. Start free, no sales call required.