Securing AI and Machine Learning Systems: A Practical Defense Blueprint for the Enterprise

A white paper by Emmanuel Naweji · November 2026

Abstract

Artificial intelligence has added three assets that classic security tools were never designed to protect: training data, trained models, and autonomous agents that act on a user's behalf. This paper proposes a five-layer defense blueprint covering data, models, the AI supply chain, applications and agents, and the infrastructure underneath. Each layer lists concrete controls and the evidence an auditor would expect to see. We pay particular attention to prompt injection and to the OWASP Top 10 for Agentic Applications (December 2025). We then map the blueprint to the NIST AI Risk Management Framework, ISO/IEC 42001, MITRE ATLAS, and the revised EU AI Act deadlines. The paper closes with a 90-day implementation roadmap, eight board-level metrics, and a teaching guide for university courses in cybersecurity, machine learning engineering, and technology management.

Keywords: AI security, machine learning security, MLSecOps, prompt injection, agentic AI, AI governance, NIST AI RMF, ISO/IEC 42001, EU AI Act

Research context: This paper is the first in a white paper series drawn from the author's doctoral research at National University. That research asks how autonomous systems can repair hospital infrastructure quickly without adding clinical or compliance risk. This paper covers the security and audit layer of that question: what it takes for AI systems and agents to be trusted enough to act.

Executive summary

An AI system is only as secure as its weakest layer, and AI brings layers that traditional security does not watch: the data, the model, and the agent that acts on its own.

A firewall cannot see a poisoned training set. A code scanner will not catch a sentence hidden in a PDF that tells an assistant to email customer records to a stranger. Access controls were built for people, not for a software agent holding 40 API keys and working at machine speed.

This paper offers security, engineering, and business leaders a single model for protecting AI systems, from the first dataset to an autonomous agent in production. It is written for teams that build AI at scale, for the executives accountable for it, and for the students who will soon do both jobs.

Readers will find	Where
A map of where attackers strike across the AI lifecycle	Section 1, Figure 1
Five layers of defense, each with controls and audit evidence	Section 2, Tables 1–5
A playbook for generative AI and autonomous agents	Section 3, Figure 3, Tables 6–7
Governance mapped to NIST, ISO, MITRE, OWASP, and the EU AI Act	Section 4, Figure 4, Tables 8–9
A 90-day roadmap and eight board metrics	Section 5, Figure 5, Table 10
Learning objectives, discussion questions, and a lab	Section 6

The bottom line: treat every model as code, every dataset as a dependency, and every AI agent as a new employee with credentials. Each one should be inventoried, tested, monitored, and governed. Organizations that build this habit now will ship AI faster, because security stops being the reason they say no.

1. Why AI security is different

Traditional software has one main attack surface, the code. An AI system has five: the data, the training process, the model, the application, and the agents. All five rest on a shared supply chain and shared infrastructure.

Figure 1. The AI attack surface: five stages resting on two shared foundations

Timing makes the problem harder. An attack early in the chain, such as poisoned data, can stay hidden until the model is in production. An attack late in the chain, such as prompt injection against an agent, can turn a helpful assistant into an insider threat within seconds.

Four reasons traditional controls miss these attacks:

Behavior changes without code changes. A model can be compromised while every line of source code stays the same.
Data now behaves like instructions. Text that a model reads can steer what it does, so every document, email, and web page becomes possible attack input.
Outputs are probabilistic. The same input can produce different answers, so one passing test proves very little.
Agents act at machine speed. They move faster than any human reviewer, with whatever access they were given.

2. The five-layer defense blueprint

Protect an AI system the way you would protect a building: secure the foundation, guard every door, watch every room, and keep the keys in a few trusted hands. Each layer below has one goal, the controls that reach it, and the evidence an auditor would ask for.

Figure 2. The five-layer AI defense blueprint, one goal per layer

No single layer is enough on its own. The value comes from stacking them, so that an attacker who slips past one control still meets the next.

Layer 1: Protect the data

Goal: only clean, approved, and lawfully obtained data trains and feeds your models.

Table 1. Data controls

Control	What it means in practice	Audit evidence
Data lineage and versioning	Record where every training and fine-tuning dataset came from and which model used it	Lineage records for each production model
Integrity checks	Hash and sign approved datasets; reject anything that changed without approval	Signed dataset manifests
Poisoning detection	Scan for outliers, flipped labels, and duplicated or injected samples before training	Pre-training data quality report
Least-privilege data access	Separate read and write access to training stores; no shared admin accounts	Quarterly access review
Privacy protection	Remove or mask personal and regulated data, such as health or financial records, before a model sees it	Data classification and masking logs
Retrieval (RAG) hygiene	Treat every document in a vector store as untrusted input, and enforce the user's own permissions at retrieval time	Permission-aware retrieval tests

Layer 2: Protect the model

Goal: the model behaves as intended, cannot be stolen, and cannot be quietly swapped.

Table 2. Model controls

Control	What it means in practice	Audit evidence
Signed model registry	Only signed, approved model versions can be deployed	Registry approval records
Adversarial and red-team testing	Test for evasion, jailbreaks, prompt injection, and data extraction before release and after major changes	Red-team report for each release
Theft protection	Rate-limit and monitor APIs for extraction patterns; encrypt weights at rest and in transit	API abuse alerts, key management logs
Safe serialization	Load models only from safe formats such as safetensors, never from untrusted pickle files	Build pipeline policy
Evaluation gates	Block deployment when accuracy, bias, or safety scores fall below agreed thresholds	CI/CD gate results

Layer 3: Protect the supply chain

Goal: know exactly what is inside each AI system, just as you know what is inside your software.

Table 3. Supply chain controls

Control	What it means in practice	Audit evidence
AI bill of materials (AI-BOM)	List every model, dataset, library, and third-party API in each AI system	An AI-BOM for each application
Vetted model sources	Approve open-source and vendor models before use and scan them for malicious code	Approved model catalog
Dependency scanning	Scan ML libraries, containers, and notebooks for known vulnerabilities	Scanner reports and patch deadlines
Vendor risk reviews	Assess AI vendors on data use, retention, training on your data, and incident response	Signed vendor assessments

Layer 4: Protect the application and its agents

Goal: what the AI says and does stays within safe limits, even under attack.

Table 4. Application and agent controls

Control	What it means in practice	Audit evidence
Input and output guardrails	Filter prompts and responses for injection attempts, sensitive data, and policy violations	Guardrail logs and block rates
Least-privilege tools	Give each agent only the tools and permissions its job needs, with short-lived credentials	Agent permission inventory
Human approval for high-impact actions	Require a person to approve payments, deletions, external emails, and production changes	Approval audit trail
Isolation	Run AI-generated code in sandboxes and separate agents by trust level	Sandbox configuration
Complete audit logging	Log every prompt, tool call, and action, along with the identity behind it	Searchable agent activity logs

Layer 5: Protect the infrastructure and operations

Goal: secure the GPUs, clusters, pipelines, and people that run AI every day.

Table 5. Infrastructure and operations controls

Control	What it means in practice	Audit evidence
Hardened ML platforms	Patch and lock down notebooks, training clusters, and inference servers; no public endpoints by default	Configuration benchmarks
Secrets management	Keep API keys out of code, notebooks, and prompts, and rotate them automatically	Secret-scanning results
Zero-trust networking	Segment training, serving, and data networks, and verify every connection	Network policies
Continuous monitoring	Watch for model drift, abuse, unusual cost spikes, and odd agent behavior	AI-aware SOC dashboards
AI incident response	Keep playbooks for model rollback, key revocation, and agent shutdown, including a tested kill switch	Tabletop exercise results

The principle behind all five layers is security by design. Controls belong inside the ML pipeline itself, a practice often called MLSecOps, so that security moves as fast as engineers ship.

3. Securing generative AI and autonomous agents

A chatbot can say something wrong; an agent can do something wrong. That difference is why agents are now the top priority in AI security.

An agent reads, decides, and acts. It calls APIs, writes code, moves files, and talks to other agents. Every tool it holds is a door an attacker can try, often just by leaving instructions where the agent will read them.

3.1 The leading threat: prompt injection

Prompt injection hides instructions inside content the model reads, such as a web page, an email, a PDF, a support ticket, or a code comment. The model cannot reliably tell the owner's instructions from the attacker's. OWASP ranks it first in its Top 10 for LLM Applications, and no single filter stops it completely.

Figure 3. How an indirect prompt injection travels, and where it can be stopped

The attacker never touches the agent directly. The attack rides in on ordinary content, so the defenses sit between each step rather than at a single front door.

The practical response is to assume injection will sometimes succeed and to limit what it can reach:

Treat all external content as data, never as commands.
Give each agent the fewest tools and the narrowest permissions possible.
Require human approval before any irreversible or high-impact action.
Never let one agent session hold both private data and an open channel to the outside world, such as sending email or fetching any URL, without a checkpoint between them.
Log every tool call and alert on unusual patterns.

3.2 The agentic threat list

In December 2025, OWASP published the Top 10 for Agentic Applications, the first industry list written specifically for autonomous agents.

Table 6. OWASP Top 10 for Agentic Applications, with a first line of defense for each

Risk	What it looks like	First defense
ASI01 Agent Goal Hijack	Hidden instructions redirect the agent's objective	Isolate untrusted content; check goals before acting
ASI02 Tool Misuse	A legitimate tool is used to cause harm	Tool allow-lists and parameter validation
ASI03 Identity and Privilege Abuse	The agent inherits or escalates excessive access	A separate identity per agent; short-lived, scoped credentials
ASI04 Agentic Supply Chain	Malicious plugins, tools, or MCP servers	A vetted, signed tool registry
ASI05 Unexpected Code Execution	The agent runs code an attacker controls	Sandboxed execution
ASI06 Memory and Context Poisoning	Bad data is planted in long-term memory or RAG stores	Memory validation and provenance tracking
ASI07 Insecure Inter-Agent Communication	Agents trust spoofed or altered messages	Authenticated, signed messaging between agents
ASI08 Cascading Failures	One agent's error spreads across the system	Circuit breakers and rate limits
ASI09 Human-Agent Trust Exploitation	People approve harmful actions because they trust the agent	Clear action previews; extra friction for high-risk steps
ASI10 Rogue Agents	An agent drifts away from its intended behavior	Behavioral monitoring and a kill switch

3.3 Treat every agent like a new employee

This is the simplest rule a leader can give an organization. No company would hand a new hire the master keys on day one, and the same logic applies to agents.

Table 7. The new-employee rule for AI agents

For a new employee	For an AI agent
A badge with their own name	Its own named identity, never a shared or human account
A job description	A written scope of what it may and may not do
Access to the systems the role needs	Only the permissions its task needs, reviewed often and revoked quickly
A manager	A named human owner who is accountable for its actions
Records of their work	A complete log of everything it did
An exit process	A tested way to shut it down immediately

3.4 A worked example: a remediation agent in a hospital

Consider an agent that restores failed services on a hospital's cloud infrastructure, such as the systems that carry bedside telemetry or send orders to a medication-dispensing robot. The controls in this paper apply as follows:

Least privilege: the agent holds two permissions only, restarting a service and checking its health. It cannot delete systems or read secrets.
Risk tiers: services are tiered by clinical impact. Tier 1 covers patient-facing systems such as telemetry and medication dispensing.
Human approval: any change to a Tier 1 service waits for an on-call engineer to approve it.
Audit trail: every action is logged with the agent's identity, the reason, and the outcome in a tamper-evident record.
Kill switch: the on-call engineer can stop the agent immediately.

Autonomy can therefore be granted in steps. The agent acts alone on low-risk systems and asks first wherever patient care is at stake.

4. Governance, regulation, and frameworks

No organization needs to invent an AI security program from scratch. The stronger approach is to build on frameworks that regulators and auditors already recognize, and to map every control to them.

Table 8. Recognized frameworks and how to use them

Framework	What it is	Use it for
NIST AI Risk Management Framework (AI RMF 1.0) and its Generative AI Profile (NIST AI 600-1)	A voluntary U.S. framework built on four functions: Govern, Map, Measure, and Manage	The backbone of an AI risk program
NIST AI 100-2 E2025, Adversarial Machine Learning	A shared vocabulary for attacks on predictive and generative AI	Threat modeling and red-team scope
ISO/IEC 42001	An international, certifiable standard for AI management systems	Proving governance to customers and auditors
MITRE ATLAS	A knowledge base of real-world adversary tactics against AI	Detection engineering and red teaming
OWASP GenAI Security Project	Top 10 lists for LLM and agentic applications	Developer standards and application testing

4.1 The regulatory clock

The EU AI Act applies to any organization whose AI systems reach people in the EU, which includes most large multinationals and many universities. The Digital Omnibus on AI (Regulation (EU) 2026/1744), in force since 27 July 2026, pushed back the main high-risk deadlines.

Figure 4. EU AI Act application dates. Sources: Regulation (EU) 2024/1689 and the Digital Omnibus on AI (Legal 500; Gibson Dunn)

Stand-alone high-risk systems, such as AI used in hiring, education, credit scoring, critical infrastructure, and law enforcement, must comply by 2 December 2027. AI built into regulated products, such as medical devices and machinery, follows on 2 August 2028.

The delay buys time; it does not remove the work. High-risk systems will still need documented risk management, data governance, logging, human oversight, and cybersecurity, and in a large organization that build takes well over a year. In the United States, healthcare and financial firms also answer to sector regulators and existing privacy and model-risk rules, which already expect much of the same discipline.

4.2 Who owns what

Table 9. Ownership of AI security across the organization

Role	Owns
Board and CEO	AI risk appetite and which uses are allowed
CISO	AI security controls, red teaming, and incident response
Chief Data or AI Officer	AI inventory, model lifecycle, and data governance
Engineering and ML leads	Secure pipelines, evaluation gates, and guardrails
Legal, privacy, and compliance	Regulatory mapping, vendor contracts, and documentation
Business owners	Each system's purpose, users, and human oversight

One rule turns governance from paper into practice: no AI system reaches production without a named owner, an inventory entry, and a risk tier.

5. The 90-day roadmap and board metrics

In 90 days, an organization can move from "we don't know what AI we run" to "every AI system is inventoried, tiered, tested, and watched." The work falls into three 30-day phases, and each phase ends with a gate that must be met before the next begins.

Figure 5. The 90-day roadmap: three phases, each closed by a gate

The order matters. Teams that start with tooling before they have an inventory usually end up protecting the wrong systems.

5.1 Metrics the board should see every quarter

Table 10. Board-level AI security metrics

Metric	Why it matters
Share of AI systems inventoried with a named owner (%)	You cannot protect what you cannot see
Share of high-risk systems red-teamed in the last six months (%)	Shows that testing keeps pace with change
Mean time to detect and contain an AI incident (hours)	Measures real readiness rather than paper policy
Share of agents with least-privilege, scoped credentials (%)	Shows how far a compromised agent could reach
Critical AI vulnerabilities open past their deadline (count)	Tracks execution discipline
Guardrail block rate and false-positive rate (%)	Shows protection without crushing usefulness
Share of AI vendors with a completed risk review (%)	Covers the part of the supply chain you do not control
Share of agent actions with a complete, tamper-evident audit record (%)	Shows that every automated action can be traced and explained to an auditor

6. Using this paper in the classroom

This paper fits one or two class sessions in an upper-level undergraduate or graduate course in cybersecurity, machine learning engineering, information systems, or technology management. No coding background is required for the discussion questions; the lab assumes basic Python.

6.1 Learning objectives

After studying this paper, students should be able to:

Explain why data, models, and agents form attack surfaces that traditional application security does not cover.
Classify a given AI attack by lifecycle stage and by defense layer.
Design least-privilege controls for an AI agent and justify each one.
Map a set of controls to the NIST AI RMF functions and to ISO/IEC 42001.
Propose metrics that let a board judge whether an AI security program is working.

6.2 Discussion questions

Prompt injection has no complete technical fix. Is it more like SQL injection, which input handling largely solved, or more like social engineering, which can only be managed? Defend your answer.
The "new employee" rule treats agents like people. Where does the analogy break down, and what risks does it hide?
The EU delayed its high-risk deadlines to 2027 and 2028. Should a U.S. organization with no EU customers still follow the Act? Why or why not?
Which of the eight board metrics in Table 10 is easiest to game, and how would you guard against that?
A hospital wants an agent that drafts discharge summaries and emails them to patients. Using Section 3, list the three controls you would insist on before launch.

6.3 Lab exercise: break and then defend a small agent (about 2 hours)

Build. Give a small LLM agent two tools: reading a folder of text files and sending a message to a mock inbox.
Attack. Plant an instruction inside one of the files that asks the agent to forward the folder's contents to an outside address. Record whether the attack works.
Defend. Add three controls from Table 4: an allow-list of recipients, human approval before sending, and a log of every tool call.
Measure. Run the attack again ten times and report how often it succeeds, plus any loss of usefulness.
Reflect. In one page, map each control to a layer in Figure 2 and to an OWASP risk in Table 6.

Instructors should run the lab in a sandbox with mock data only, never with real accounts or personal information.

7. Conclusion: security is how AI scales

The organizations that win with AI will not simply be the fastest. They will be the ones that move quickly and can still be trusted.

Every major technology wave, from the web to the cloud, has followed the same pattern: adoption first, breaches next, and discipline last. AI is moving faster than any wave before it, and agents now give software the power to act on its own. Waiting for a breach is the most expensive way to learn that lesson.

Start with three moves this quarter. Inventory every AI system. Give each one an owner and a risk tier. Treat every agent like a new employee with credentials. The rest of this blueprint builds on those three steps.

References

European Parliament and Council of the European Union. (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). EUR-Lex
Gibson Dunn. (2026). EU AI Act omnibus agreement: Postponed high-risk deadlines and other key changes. Gibson Dunn
International Organization for Standardization. (2023). ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system. ISO
Legal 500. (2026). EU Digital Omnibus on AI enters into force. Legal 500
MITRE. (n.d.). ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems. MITRE ATLAS
National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0) (NIST AI 100-1). NIST
National Institute of Standards and Technology. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1). NIST
National Institute of Standards and Technology. (2025). Adversarial machine learning: A taxonomy and terminology of attacks and mitigations (NIST AI 100-2 E2025). NIST CSRC
OWASP GenAI Security Project. (2025). OWASP Top 10 for LLM Applications 2025. OWASP
OWASP GenAI Security Project. (2025, December 9). OWASP Top 10 for Agentic Applications for 2026. OWASP

Suggested citation

Naweji, E. (2026). Securing AI and machine learning systems: A practical defense blueprint for the enterprise [White paper]. Transformed 2 Succeed. https://www.emmanuelnaweji.com

About the author

Emmanuel Naweji is an engineer who builds enterprise systems for the public sector and a PhD candidate in computer science at National University researching agentic AI and quantum-inspired optimization for resilient health-system infrastructure. He founded Transformed 2 Succeed (T2S), which trains engineers in AI and ML systems, DevSecOps, site reliability engineering, and FinOps. He served as a full-time pastor for more than 14 years and now serves God through Kids Teck Inc, Transformed 2 Succeed, and other projects he is building. Through Kids Teck Inc, he is helping young people access technology and compete with their peers anywhere in the world. He believes complex technology should be simple enough for anyone to understand and secure enough for everyone to trust.

To bring this blueprint to your team or classroom, book a call at transformed2succeed.com.

This white paper is for general information only and is not legal advice. Consult qualified counsel about specific regulatory obligations.