AI Security and AI Governance: Protect Every AI Component

AI security is the practice of protecting every part of an AI system, from the data it learns from to the tools it can act with, against attack and misuse. AI governance is the set of policies, roles, and oversight that decide how AI may be used in your organization and that prove those decisions are being followed. Security without governance leaves nobody accountable; governance without security is a document that attackers ignore. You need both.

Why AI needs its own security approach

Traditional controls still matter, but AI adds properties they were not built for:

• Instructions and data share one channel: a language model cannot reliably tell a trusted instruction from text it merely read in an email, web page, or document. This is why prompt injection is LLM01 in the OWASP Top 10 for LLM Applications 2025
• Behavior is probabilistic: the same input can produce different outputs, so you cannot test every case the way you test a function
• The attack surface is wider than code: training data, embeddings, model files, and even tool descriptions can be tampered with
• Text becomes action: once a model can call tools, a manipulated answer can turn into a deleted file, a sent email, or a payment (OWASP LLM06, Excessive Agency)

NIST's adversarial machine learning taxonomy (AI 100-2, 2025 edition) groups attacks into evasion, poisoning, privacy, and abuse, and MITRE ATLAS catalogs real adversary techniques against AI-enabled systems. The seven layers below turn those lists into something you can assign to owners.

The seven layers of AI to protect

Think of an AI deployment as seven layers. A gap in any one of them can undo the controls in the others.

• 1. Data: training, fine-tuning, and retrieval data. Risks: poisoning (LLM04) and leakage of sensitive information (LLM02). Controls: classify data before it reaches a model, minimize or anonymize personal data, track provenance, restrict who can write to training and knowledge sources, and check integrity
• 2. Model and supply chain (LLM03): pretrained models, datasets, libraries, and plugins you did not build. Controls: use trusted registries, pin versions and verify hashes, prefer safe model formats such as safetensors over pickle-based files that can execute code when loaded, scan and sandbox before use, and keep an inventory of components (an AI bill of materials)
• 3. Prompts and outputs (LLM01, LLM05, LLM07): Controls: keep secrets out of system prompts, since they can leak; filter inputs and outputs as one defensive layer, never the only one; and treat every model output as untrusted input to the next system, so validate, encode, or parameterize it before it reaches a database, shell, or browser
• 4. Retrieval and vector stores (LLM08): RAG indexes often hold your most sensitive documents. Controls: enforce each user's permissions at retrieval time, isolate tenants, vet what gets ingested because a poisoned document is an injection vector, and protect embeddings like the source data
• 5. Agents and tools (LLM06): Controls: least privilege per tool, short-lived scoped credentials tied to the real user, human approval for high-impact actions such as payments, deletions, and external messages, and full logging of every tool call. See What Is MCP? Architecture, Security Risks, and Technical Defenses for the technical detail
• 6. Infrastructure and identity (LLM10): API keys, endpoints, GPUs, and pipelines. Controls: secrets management, strong authentication on every model endpoint, network segmentation, rate limits and spending quotas to stop unbounded consumption, and encryption in transit and at rest
• 7. People, usage, and monitoring (LLM09): Controls: approved tools so staff do not drift into Shadow AI, training on what data may go where, human review where wrong answers cause harm, logging with privacy in mind, detection, regular red teaming, and an incident playbook

A useful habit is to ask the same four questions of every AI use case: what data goes in, who can influence the input, what can the output trigger, and who is told when it goes wrong. If any answer is "nobody knows", that is your first finding.

What AI governance actually consists of

AI governance is not a single policy. Mature programs share a small set of working parts:

• Accountability: a named executive owner, a small cross-functional group (security, legal, privacy, data, and the business), and a named risk owner for every AI system
• AI inventory: a living list of every AI system, including AI features embedded in SaaS tools, vendor models, and unsanctioned use. You cannot govern what you cannot see
• Policy set: an acceptable-use policy (which data classes may go into which tools), plus procurement, data, human-oversight, and incident policies
• Risk tiering: classify use cases by the impact of a wrong or manipulated output, not by the technology, and scale review to the tier
• Vendor due diligence: ask whether your data is retained or used for training, where it is processed, which sub-processors are involved, what certifications exist (such as ISO/IEC 42001, ISO/IEC 27001, SOC 2), how quickly incidents are reported, and how you exit
• Testing and assurance: pre-deployment evaluation, red teaming, and re-testing whenever the model, prompt, or data source changes
• Monitoring and audit: logs, risk indicators, periodic reviews, and evidence that you can show a regulator or customer

Risk tiering is the part that keeps governance proportionate. A summarizer for public documents and an agent that can change production systems should not face the same process.

Anchor on existing frameworks

You do not need to invent your own. Each of these answers a different question:

• NIST AI RMF (Govern, Map, Measure, Manage) and the Generative AI Profile (NIST AI 600-1): a voluntary risk-management structure. See What Is the NIST AI RMF? A Practical Guide to AI Risk Management
• ISO/IEC 42001 (published December 2023): a certifiable AI management system standard, useful when customers or auditors ask for independent assurance
• OWASP Top 10 for LLM Applications 2025: the technical threat list used in the layers above
• MITRE ATLAS and NIST AI 100-2: adversary techniques and attack taxonomy for threat modeling and red teaming
• NCSC Guidelines for Secure AI System Development: lifecycle guidance across design, development, deployment, and operation, jointly published with CISA and international partners

Regulation snapshot (as of October 2026)

Regulation changes quickly, so confirm current status before relying on any date:

• EU AI Act: risk-based, with extraterritorial reach. According to law-firm summaries, the Digital Omnibus on AI (Regulation (EU) 2026/1744, in force since July 27, 2026) moved the high-risk obligations for Annex III systems to December 2, 2027 and for Annex I product-embedded systems to August 2, 2028, while transparency duties and general-purpose AI model obligations were not postponed
• Thailand: as far as public sources show, the Draft AI Act is still a draft after its 2026 public consultation. Until a law is enacted, AI use is governed indirectly through the PDPA, consumer protection rules, and sector regulators, so any AI that processes personal data already falls under the PDPA

A governance program built on risk tiers, an inventory, and evidence maps well onto either regime, so you will not need to start over when a law lands.

Start today

You can make real progress in a few weeks without buying anything:

• Build the inventory: survey teams, check SaaS and network logs, and include embedded AI features
• Publish a one-page acceptable-use policy: a table of data classes against approved tools is enough to start
• Name an owner and a group: accountability matters more than committee size
• Tier your use cases: require security and privacy review for the top tiers only
• Apply the highest-value controls first: least-privilege agents, output validation, permission-aware retrieval, secrets management, and rate limits
• Log, test, and rehearse: schedule a red-team exercise and write an AI-specific incident playbook

To see where you stand, try the self-assessment, which maps your answers to prioritized next steps. For context on how AI also strengthens defense, read Using AI in the SOC to Reduce MTTD and MTTR.

Common mistakes

• Treating guardrails as the control: filters and system prompts reduce risk but can be bypassed, so back them with permissions, validation, and approvals
• Governance on paper only: a policy nobody enforces or measures is a liability, not a control
• Banning AI outright: it usually pushes use underground and creates Shadow AI. Provide approved options instead
• Trusting a vendor promise alone: "we do not train on your data" should be backed by contract terms and evidence
• Assessing once: models, prompts, and data sources change, so reassess on change as well as on a schedule

The agents-and-tools layer needs deeper controls: autonomy levels, trust between agents, and emergency shutdown. Continue with Agentic AI Security: Controlling AI That Acts on Its Own.

A concrete case of AI governance is coding assistants and vibe coding. See AI Coding Assistant Security and Vibe Coding Risks.

For red teaming an AI system step by step, with reference frameworks, continue with AI Red Teaming: How to Test AI System Security.

References and further reading

• OWASP Top 10 for LLM Applications 2025: the ten risks used above
• NIST AI Risk Management Framework and the Generative AI Profile (NIST AI 600-1)
• NIST AI 100-2 E2025: Adversarial Machine Learning taxonomy
• MITRE ATLAS: adversary tactics and techniques against AI systems
• ISO/IEC 42001: AI management systems
• NCSC: Guidelines for secure AI system development
• Google Secure AI Framework (SAIF)
• EU AI Act regulatory framework and Orrick: Digital Omnibus changes
• Baker McKenzie: Thailand Draft AI Act, seven key implications