AI Governance Glossary

41+ terms, standards-grounded and practitioner-tested

Definitions your compliance team, legal counsel, and board can rely on, grounded in the standards and US laws that govern enterprise AI programs.

A

Agentic AI: An AI system that autonomously executes multi-step workflows, calls external tools or APIs, and makes sequential decisions with limited or no human intervention between steps. Agentic systems amplify AI capability and risk simultaneously: errors compound across steps, and the blast radius of a failure is larger than for a single-turn query. Governance programs need specific policies for human-in-the-loop thresholds, tool-call scope, and audit logging.
AI Center of Excellence (CoE): A cross-functional team, typically spanning IT, legal, risk, compliance, HR, and business lines, responsible for defining AI strategy, setting governance standards, reviewing use cases, and managing the model portfolio across the organization. The CoE is the organizational structure that makes a governance framework operational rather than theoretical.
AI Governance Framework: The documented set of policies, standards, processes, and accountabilities that an organization uses to manage the full lifecycle of its AI systems, from use-case ideation and pre-deployment risk assessment through post-deployment monitoring and decommissioning. A governance framework is distinct from an AI strategy (which answers 'why and where') and an AI architecture (which answers 'how it works technically').
AI Incident Response: The documented process for detecting, classifying, containing, investigating, and remediating AI-related failures, including model errors, biased outputs, security breaches involving AI systems, and regulatory violations. An AI incident response plan supplements (and references) the organization's existing cyber incident response plan rather than replacing it.
AI Literacy: The knowledge, skills, and attitudes required for individuals to use AI tools responsibly and to understand the capabilities, limitations, and risks of the AI systems they interact with. The EU AI Act (Article 4) requires providers and deployers to take measures to ensure sufficient AI literacy of their staff. Documented training programs and completion records are the expected evidence.
AI Management System: An ISO 42001 concept: the set of interrelated elements (policies, objectives, processes, documented information, resources, and organizational roles) that an organization uses to manage its AI-related activities. ISO 42001 uses this term the same way ISO 27001 uses 'Information Security Management System': as the structured, auditable whole that a certification body assesses.
AI Risk Management Framework (NIST AI RMF): The voluntary framework published by NIST in January 2023 organizing AI risk management around four functions: GOVERN (establish culture and accountability), MAP (identify context and risks), MEASURE (analyze and assess risks), and RESPOND (prioritize and manage risks). The companion Playbook provides concrete suggested actions for each sub-category. The AI RMF is the most widely referenced AI governance standard in the US.
AI Risk Tiering: The practice of classifying AI use cases by their potential for harm, typically Prohibited, High, Medium, and Low, so governance requirements can be calibrated to actual risk rather than applied uniformly. The EU AI Act codifies risk tiering into law; NIST AI RMF's MAP function treats it as a prerequisite for any meaningful risk measurement.
AI System Inventory: A register of all AI systems and models deployed or in development within an organization, including metadata on purpose, data sources, model type, risk classification, owner, and deployment status. Maintaining a current inventory is a baseline requirement under NIST AI RMF (GOVERN 1.1), ISO 42001 (Clause 8.4), and the EU AI Act (Article 49 for high-risk systems).
Algorithmic Bias: Systematic, repeatable error in an AI system's outputs that creates unfair or discriminatory outcomes for individuals or groups, often correlated with protected characteristics (race, sex, age, disability). Bias can enter through training data (historical discrimination reflected in labels or features), model architecture choices, or deployment context mismatches. The EU AI Act's non-discrimination requirements and US state AI laws (Colorado SB 205) create legal obligations around bias detection and remediation for high-risk use cases.
Algorithmic Impact Assessment (AIA): A structured analysis, conducted before deploying an AI system into a consequential use case, that documents the system's purpose, data sources, potential for discriminatory impact, affected populations, and planned mitigation measures. The EU AI Act requires a conformity assessment (a related concept with a more formal legal structure) for high-risk systems. US state laws (Colorado, Connecticut) require algorithmic impact assessments for automated decision-making in covered categories.

B

Bias Audit: An independent evaluation of an AI system's outputs to detect disparate impact or discriminatory patterns across demographic groups. New York City Local Law 144 (effective since 2023) requires annual bias audits for automated employment decision tools used in the city. Bias audits differ from internal bias testing in that they are typically conducted by an independent third party and the results are published.

C

Conformity Assessment: Under the EU AI Act, the process by which a provider of a high-risk AI system demonstrates compliance with the Act's requirements before placing the system on the market. For most Annex III high-risk systems, this is an internal assessment; for certain categories (biometrics, critical infrastructure), third-party assessment by a notified body is required.

D

Data Provenance: The documented record of a dataset's origin, transformations, and chain of custody: who collected it, from where, under what legal basis, how it was cleaned, labeled, and versioned, and how it moved from source to training pipeline. Regulators and auditors increasingly require provenance documentation as evidence that training data did not introduce bias, violate privacy law, or infringe on intellectual property.

E

EU AI Act: Regulation (EU) 2024/1689 of the European Parliament and of the Council, published in July 2024. The first binding AI-specific legislation from a major regulatory body, it establishes a risk-tiered framework: prohibited practices (banned outright), high-risk systems (Annex III, conformity requirements), GPAI models (Articles 53–55, transparency and testing obligations), and limited/minimal risk categories. Enforcement is staggered, with the prohibition on banned practices applying from February 2025.
Explainability: The degree to which an AI system's outputs, decisions, or recommendations can be understood and articulated in human terms, either by inspecting the model's internal workings (interpretability) or by generating post-hoc explanations of individual predictions (explainability). Regulations and governance standards increasingly require explainability for high-stakes decisions, though the specific requirement varies (GDPR Article 22 right to explanation, EU AI Act Article 13 transparency).

F

FedRAMP: The Federal Risk and Authorization Management Program, which standardizes security assessment, authorization, and continuous monitoring for cloud services used by US federal agencies. AI systems offered as cloud services to federal agencies must maintain FedRAMP authorization at the appropriate impact level (Low, Moderate, or High). The relevant NIST 800-53 control families for AI systems include AC, AU, CA, CM, IR, RA, SA, and SI.
Foundation Model: A large AI model trained on broad data at scale and adapted (fine-tuned, prompted, or retrieved against) for a wide variety of downstream tasks. The term was coined by the Stanford Center for Research on Foundation Models in 2021. The EU AI Act uses the term 'General-Purpose AI (GPAI) model' for the same concept and imposes specific obligations (Articles 53–55) on providers of models above defined compute and capability thresholds.

G

General-Purpose AI (GPAI) Model: The EU AI Act's term for an AI model trained on large amounts of data with broad capability that can be adapted to a wide range of downstream tasks. GPAI model providers must maintain technical documentation, comply with EU copyright law for training data, and publish a summary of training data. GPAI models with 'systemic risk' (above 10^25 FLOPs or otherwise designated) face additional obligations including adversarial testing.
Governance Council: The senior cross-functional committee, typically chaired by a Chief AI Officer, CTO, or Chief Risk Officer, that oversees the AI governance program, reviews high-risk use cases, sets AI risk appetite, and receives regular reporting from the AI Center of Excellence. Distinct from the CoE (which does the operational work) in the same way a board of directors is distinct from management.

H

Hallucination: The tendency of large language models (LLMs) to generate outputs that are confident in tone but factually incorrect, fabricated, or not grounded in the model's training data. Hallucination is not a bug to be fixed in a single release; it is a systematic property of probabilistic next-token prediction. Governance controls (human review gates, retrieval-augmented generation, confidence thresholds) can reduce, but not eliminate, hallucination risk in high-stakes applications.
High-Risk AI System: Under the EU AI Act, an AI system listed in Annex III: systems used in biometric identification, critical infrastructure management, education and vocational training, employment and worker management, access to essential services (credit, insurance), law enforcement, migration and border management, or administration of justice. High-risk systems face the most extensive compliance requirements: conformity assessment, transparency, accuracy, robustness, logging, and human oversight obligations.
Human-in-the-Loop (HITL): A design requirement that a human reviewer must confirm, override, or approve an AI system's output before it takes effect. The appropriate HITL threshold depends on the stakes of the decision: a marketing email draft might require no human review; a credit denial, a patient discharge decision, or an autonomous agent's tool call requires escalating degrees of oversight. EU AI Act Article 14 codifies human oversight as a required characteristic of high-risk systems.

I

ISO/IEC 42001: The international standard for AI Management Systems, published in December 2023. Structured on the Annex SL framework used by ISO 27001 (information security) and ISO 9001 (quality), it provides a certifiable governance structure for organizations that develop, provide, or use AI systems. Annex A provides a controls catalogue; Annex B maps controls to organizational contexts; Annex C maps controls to AI system impacts.

M

Model Card: A concise, structured document that describes a machine learning model: its intended use, out-of-scope applications, training data, performance metrics across demographic slices, ethical considerations, and limitations. Originally proposed by Google researchers (Mitchell et al., 2019). The EU AI Act requires similar technical documentation for high-risk systems (Article 11); NIST AI RMF's GOVERN function recommends model cards as a transparency mechanism.
Model Drift: The degradation of an AI model's performance over time as the statistical relationship between inputs and outputs shifts away from the patterns the model learned during training. Data drift (input distribution changes) and concept drift (the underlying relationship changes) are the two main types. Post-deployment monitoring programs must define drift detection thresholds and escalation paths before a model goes live.
Model Lifecycle Management: The end-to-end operational process for an AI model from initial development through deployment, monitoring, retraining, and eventual retirement. A mature model lifecycle program tracks each model in the inventory register, defines thresholds for performance review and retraining, and maintains audit logs sufficient to reconstruct model state at any point in time.
Model Risk Management (MRM): A risk management discipline developed in financial services (originating with the Federal Reserve's SR 11-7 guidance) that treats quantitative models, including AI/ML models, as sources of material risk requiring independent validation, ongoing monitoring, and management oversight. SR 11-7's core principles (development, validation, use) have been adopted by regulators beyond banking and are referenced by FINRA and OCC guidance on AI in financial services.

N

NIST AI RMF: See AI Risk Management Framework (NIST AI RMF).
Notified Body: Under EU product safety law (including the AI Act), an independent conformity assessment organization designated by an EU member state to conduct third-party assessments. For high-risk AI systems in certain Annex III categories, a notified body must certify conformity before the system can be placed on the EU market. For other Annex III systems, internal conformity assessment is permitted.

P

Post-Deployment Monitoring: The ongoing process of measuring an AI system's performance, fairness, and behavior against pre-defined thresholds after it goes live. Monitoring should cover model accuracy, data drift, output anomalies, user feedback, and adverse event rates. The EU AI Act (Article 72) requires high-risk system providers to implement post-market monitoring plans.

R

RACI Matrix: A responsibility assignment tool that maps roles against activities using four categories: Responsible (does the work), Accountable (ultimately answerable, signs off), Consulted (input required before action), and Informed (notified of outcomes). AI governance programs use RACI matrices to make accountability for model lifecycle tasks, incident response, risk review, and policy exceptions explicit and auditable.
Red-Teaming: Adversarial testing in which a team attempts to elicit harmful, biased, or policy-violating outputs from an AI system. For GPAI models, the EU AI Act (Article 55) requires adversarial testing against identified systemic risks. The NIST AI RMF Playbook recommends red-teaming as a MEASURE function action. Red-team exercises should be documented and findings remediated before deployment.
Responsible AI: An umbrella term for the design, development, and deployment of AI systems in ways that are safe, fair, transparent, accountable, and aligned with human values. Unlike 'AI ethics' (which can be philosophical), 'responsible AI' usually implies operationalized principles with measurable criteria, governance processes, and accountability structures, not just aspirational commitments.
Risk Appetite Statement: A board-level declaration of the types and amounts of risk the organization is willing to accept in pursuit of its strategic objectives. An AI-specific risk appetite statement defines, for example, the organization's tolerance for model errors in high-stakes decisions, the maximum number of ungoverned AI tools in use at one time, and the conditions under which a use case requires board-level approval.

S

Shadow AI: AI tools and applications that employees use without organizational knowledge, procurement, security review, or policy coverage, analogous to 'shadow IT.' Shadow AI is common because consumer AI tools are accessible, immediately useful, and free at point of use. The risk is that sensitive data (customer PII, proprietary business logic, regulated financial information) is processed through tools that have not been assessed for data retention, model training, or export controls compliance.
SOC 2 Type II: A trust-services audit performed by a licensed CPA firm that evaluates whether a service organization's security, availability, processing integrity, confidentiality, and privacy controls were operating effectively over a defined period (typically 6–12 months). Type II opinions carry more weight than Type I (which only covers design at a point in time). AI systems in scope for a SOC 2 audit generate operational evidence requirements for audit logging, incident response, and access control.
Systemic Risk (GPAI): Under the EU AI Act, a GPAI model is deemed to present systemic risk if it has high-impact capabilities or was trained using more than 10^25 floating-point operations (FLOPs). Models with systemic risk face additional obligations: adversarial testing, incident reporting to the EU AI Office, cybersecurity measures, and energy consumption reporting.

T

Traceability: The ability to reconstruct the complete chain of decisions, data, and processes that produced a specific AI output. Regulators and auditors require traceability to investigate adverse events, verify that governance controls were applied, and demonstrate accountability. Operationally, traceability requires audit logging at model inference time, version-controlled training pipelines, and documented decision records for high-stakes outputs.
Transparency (AI): The quality of an AI system's operation and governance being understandable and verifiable by stakeholders: operators, affected individuals, regulators, and the public. The EU AI Act (Article 13) requires high-risk systems to be designed so operators can understand their outputs; Article 50 requires disclosure to individuals who interact with certain AI systems (e.g., chatbots). Transparency is distinct from explainability: a system can be transparent about its limitations without being mathematically interpretable.
Trustworthy AI: The European Commission's High-Level Expert Group on Artificial Intelligence defined trustworthy AI through three components: (1) lawful, complying with all applicable laws and regulations; (2) ethical, adhering to ethical principles and values; and (3) robust, both from a technical perspective (reliable, secure) and social perspective (not causing unintended negative impacts). The seven key requirements: human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental well-being; accountability.

Missing a term?

The glossary is updated when a new standard or regulatory concept requires a clear definition for compliance teams. If you’re working on a governance program and want to discuss terminology or how the framework addresses a specific requirement, reach out to our team.

About this framework →Regulatory Pulse →Browse the content library →