Understanding AI model security scanning
AI model security scanning is the process of checking your models and their surrounding stack for security issues across the entire lifecycle. This includes static artifact scanning (model files, dependencies, serialization formats), pipeline and policy enforcement (CI/CD gates, admission controls), dynamic endpoint testing (prompt injection, abuse patterns), and runtime monitoring (behavioral anomalies, drift detection). You examine the model artifact, training data, inference endpoints, and cloud infrastructure that hosts them.
AI models in the cloud face some specific threats you do not see with normal apps. You will run into terms like model extraction, data poisoning, and prompt injection—attack patterns that are on the rise according to a recent Gartner survey. These threats target different stages of the AI lifecycle, from training through production inference.
Model extraction: An attacker copies your model's behavior or parameters by sending many crafted queries.
Data poisoning: Malicious data is slipped into training or fine‑tuning so the model behaves incorrectly or carries a hidden backdoor.
Prompt injection and adversarial inputs: Inputs are crafted to override instructions, leak data, or force unsafe behavior. Red-teaming research (for example, this study) demonstrates widespread policy violations against AI agents, showing that adversarial prompts can bypass safety guardrails across multiple model architectures.
Traditional security tools focus on servers and networks. They rarely understand model serialization attacks (malicious code hidden in model files), exposed training datasets, membership inference (determining if specific data was in the training set), or model inversion (reconstructing original training data from outputs).
You should think in terms of toxic combinations, not single findings. For example, a model vulnerability becomes serious when:
The model can be called from the internet.
The model has access to sensitive data stores.
The model host has overly broad permissions in your cloud account.
In practice, that means your top risks live where vulnerable models, powerful identities, and important data all meet. One global services company discovered LLM vulnerabilities this way and used automated routing to send issues to the exact teams that owned each model, instead of forcing security to chase owners manually.
GenAI Security Best Practices Cheat Sheet
This cheat sheet provides a practical overview of the 7 best practices you can adopt to start fortifying your organization’s GenAI security posture.
Implementing comprehensive model discovery and inventory
Model discovery is finding every AI asset in your environment. Model inventory is keeping an up‑to‑date record of those assets, what they touch, and who owns them.
You want this inventory to include managed services, self‑hosted models, and anything experimental. That means pulling in:
Managed endpoints from platforms like SageMaker, Azure ML, or Vertex AI.
Custom deployments on Kubernetes, VMs, or serverless.
Training infrastructure, feature stores, and batch inference jobs.
Once you know where models live, connect them to how they were built. This is model lineage: tracing a model from training data, through pipelines, into production endpoints.
Model registry: Use or create a central place to register every production‑grade model.
AI pipeline inventory: Map code repos, training jobs, evaluation steps, and deployment jobs.
Ownership: Tag models with team names, services, and business units so issues can be assigned instantly.
You also need to surface shadow AI models. These are models that teams stand up on their own, often outside official review, but still sitting in your cloud accounts—an important task given AI workloads grew rapidly year over year in the latest Sysdig usage report.
A data‑driven company that went through several mergers used this approach to scan AI usage across six different environments. With a single inventory and security graph, they could finally see which models used sensitive data and which were exposed to the internet, regardless of which original company built them.
Automating model scanning in CI/CD pipelines
Model scanning in CI/CD means you treat models like any other artifact in your build pipelines. You scan them before they move to staging or production, not only after they are live.
Start by adding model security checks as explicit pipeline steps. When training or fine‑tuning completes and a model file is produced, you:
Scan model artifacts for unsafe deserialization and embedded code. Prefer safer formats like safetensors over pickle, and block loading untrusted custom code. Pickle files can execute arbitrary Python code during deserialization, creating a direct code execution risk.
Check the environment and containers for known vulnerabilities and misconfigurations.
Generate an SBOM for AI that lists every framework and dependency involved.
Then enforce rules with policy‑as‑code. For example, you might codify: No production model can reference a development data store. A unified policy engine that spans code, pipelines, cloud, and runtime helps teams block risky AI deployments early and keep policy consistent across environments, reducing the chance that a model slips through with dangerous configurations.
No production model can reference a development data store.
No deployment is allowed if critical vulnerabilities exist in ML frameworks.
No public endpoint is allowed for specific projects or data types.
On Kubernetes, add admission controllers like OPA Gatekeeper or Kyverno as a last gate. These controllers intercept deployment requests and enforce policies before resources are created in the cluster.
Model signing: Require signed model artifacts and verify signatures before serving.
IaC scanning: Validate Terraform, Helm, and other templates that stand up AI infrastructure.
Non‑compliant deployments: Fail fast and surface clear messages to the owning team.
One global product company uses this pattern to catch misconfigurations before cloud resources are created. You can apply the same approach so any attempt to scan AI models and infrastructure happens automatically, on every change.
Securing cloud AI service deployments
Securing AI services in the cloud is about how you expose and wire up your models. You can have a perfectly safe model file and still create a big model security problem with the wrong deployment choices.
Focus first on how endpoints are exposed. Ask simple questions like: "Who can call this?" and "From where?"
Endpoint exposure: Prefer private endpoints over public ones whenever you can. Use AWS PrivateLink for SageMaker endpoints, Azure Private Link for Azure ML, or GCP Private Service Connect for Vertex AI. This keeps inference traffic within your VPC and off the public internet.
Network controls: Use private networking constructs like AWS PrivateLink, Azure Private Link, or GCP Private Service Connect to keep inference endpoints off the public internet. Service meshes and internal gateways add additional policy enforcement layers.
Then lock down authentication and permissions. Every call to a model should be authenticated, and the serving environment should follow least privilege.
Authentication: Use strong auth patterns like workload identity or short‑lived tokens. Machine identities far outnumber human identities in cloud environments and are often riskier due to excessive and unused permissions that accumulate over time.
Permissions: Ensure model hosts can only read the specific buckets, queues, or secrets they truly need.
You also need to watch for configuration drift. Dev might be using a locked‑down config while production quietly drifts into "open to the world" territory.
Drift checks: Regularly compare dev, staging, and prod for endpoint exposure, encryption, logging, and identity settings.
Encryption: Make sure model files and training data are encrypted at rest using customer-managed encryption keys (CMEK) through AWS KMS, Azure Key Vault, or GCP Cloud KMS. Protect traffic in transit with TLS 1.3 for all inference endpoints and data transfers.
When you secure deployments this way, model security stops being just about the model and becomes part of your broader cloud hygiene.
Sample AI Security Assessment
Get a glimpse into how Wiz surfaces AI risks with AI-BOM visibility, real-world findings from the Wiz Security Graph, and a first look at AI-specific Issues and threat detection rules.
Runtime model protection and monitoring
Runtime protection is how you watch models once they are actually serving traffic. Here you care less about properties of the model file and more about behavior.
Start by deploying runtime sensors on your model‑serving infrastructure, or enable cloud‑native telemetry where sensors aren't feasible. These lightweight eBPF‑based components can see:
Suspicious processes starting on model hosts.
Unexpected file writes or network connections.
Container escapes or privilege escalation attempts.
At the same time, you monitor model behavior. Unusual patterns in requests or responses often point to attacks or misuse. Bringing runtime telemetry together with cloud identity, network exposure, and data sensitivity context cuts false positives and speeds triage—you focus on anomalies that actually matter instead of chasing every outlier.
Behavioral analytics: Look for odd spikes, strange prompt patterns, or heavy probing of edge cases.
Data exfiltration: Watch for attempts to send data from model hosts to destinations you do not recognize.
You also track drift. When model outputs or input distributions change in ways you did not expect, that could be a sign of poisoning, broken upstream systems, or a quiet attack.
A software company that relies on Kubernetes for critical workloads uses runtime sensors to protect containerized services. The same sensors now help them catch AI workloads that suddenly begin making outbound calls or behaving in ways that do not match normal inference patterns, giving them early warning before damage spreads.
Managing model supply chain security
Model supply chain security is about where your models and ML components come from. It is easy to pull in a powerful open‑source model and accidentally pull in a hidden backdoor too.
You should verify the origin and integrity of all third-party and open-source models you use. That includes pre‑trained weights, fine‑tuning checkpoints, and even small helper models.
Model provenance: Record where each model came from and how it was changed.
Repository scanning: Scan model repositories for malicious artifacts or known unsafe formats. Enforce model signing and verify signatures before use to ensure models haven't been tampered with between training and deployment.
Next, maintain a clear software bill of materials for each model. This SBOM should list frameworks, libraries, and system components the model depends on.
Dependency scanning: Continuously check this list against vulnerability feeds.
Regular updates: Plan and test updates to critical ML frameworks when vulnerabilities appear.
This is how you avoid a quiet model vulnerability from an upstream package turning into a full compromise. You treat your AI model supply chain with the same care as your container or operating system supply chain, with extra attention on model tampering and supply chain attacks that are unique to AI.
Governance and compliance frameworks for AI models
AI model governance is the set of rules you establish for how models are built, deployed, and used. Compliance is how you prove you followed those rules.
You start by defining policies for the model lifecycle. These policies should cover access, approvals, and when models must be retired.
Access policies: Who can see training data, model weights, and inference logs.
Usage policies: Where a model can be used, for which users, and with which data types.
Retirement policies: When a model must be taken out of service and what happens to its data.
You then define approval workflows. High‑impact or high‑risk models should go through a more formal review process before going to production.
You also map your AI controls to external and internal standards like NIST AI RMF 1.0, ISO/IEC 42001, and applicable regulations such as the EU AI Act. Align supporting controls with SOC 2 and ISO 27001 where relevant to demonstrate comprehensive security governance.
AI governance: Document who signed off on the model, what testing was done, and how risks were assessed.
Model risk assessment: Classify models into risk categories based on the data they use and the impact of failure.
Compliance mapping: Tie your model controls back to security and privacy frameworks your organization follows. Map model access controls to NIST AI RMF's Govern function, training data protection to ISO/IEC 42001's data governance requirements, and runtime monitoring to SOC 2 Type II continuous monitoring controls. For regulated industries, align with sector-specific requirements like HIPAA for healthcare AI or PCI DSS for payment processing models.
By doing this, you turn model security from a set of ad hoc decisions into a repeatable process that legal, risk, and security teams can all understand.
Incident response for compromised AI models
AI incident response is what you do when something goes wrong with a model. You handle it like any other incident, but with extra focus on models, data, and pipelines.
You begin by writing AI‑specific playbooks. These playbooks should cover things like model theft, model misuse, and adversarial attacks.
A model‑serving endpoint is behaving oddly or serving unexpected outputs.
You discover a poisoned model or dataset in your environment.
Someone has copied or downloaded model weights without approval.
Each playbook should spell out clear containment steps. For example, you might:
Rate‑limit or disable specific inference endpoints.
Switch traffic to a previous safe model version.
Cut off suspicious identities or networks from calling the model, and immediately rotate or revoke impacted credentials, tokens, and API keys. This prevents attackers from maintaining access through compromised authentication material.
You also need model forensics. This is how you investigate what happened, using logs, model lineage, and environment data.
Forensics: Capture model artifacts, logs, and relevant data for analysis.
Attack path analysis: Trace how the attacker moved from model or library issues into broader cloud resources.
A financial services company that faced a critical vulnerability in an AI‑related library used this kind of visibility to find exactly which workloads used the bad component and fix them fast. That same pattern works for any AI‑related security event.
Advanced model security techniques
Advanced model security controls are extra layers you add when the stakes are high. They are not the first thing you build, but they become important for sensitive workloads.
A few common ones:
Model watermarking: You add an invisible signal or behavior to a model so you can later prove it is yours or detect copies.
Differential privacy: You train models in a way that limits what an attacker can learn about any individual data point, even with access to outputs.
Homomorphic encryption: You structure data and computation so certain inferences can happen on encrypted data, reducing exposure of raw values.
Adversarial training: You intentionally train models on adversarial examples so they learn to resist those attack patterns.
These tools can help against model theft, training data leakage, and adversarial attacks. They come with trade‑offs in performance and complexity, so you apply them where the risk profile truly justifies the cost.
Building a unified AI security program with Wiz
A unified AI security program is when AI security and cloud security share the same platform, data, and workflows. You stop running a separate "AI security" island and bring it into your existing operating model. This approach aligns with the OWASP Top 10 for LLM Applications, helping you address the most critical security risks to AI systems in a structured way.
You start by putting all your assets into one view. That includes models, data stores, compute, identities, pipelines, and endpoints.
Security graph visualization: Use a security graph to show how a given model connects to training data, production data, identities, and networks. Graph context across models, data stores, identities, and endpoints makes it easy to trace ownership and route issues to the right team fast, so security doesn't become a bottleneck.
Contextual prioritization: Rank issues based on real attack paths, not just raw severity scores.
You then wire up automation. When model vulnerabilities, misconfigurations, or data exposures are found, they should flow to the right teams automatically.
Automated remediation: Create rules that open tickets, send alerts, or trigger pipeline failures when certain conditions are met. With AI Remediation 2.0, you get AI-powered suggestions that recommend specific fixes tailored to your environment, speeding up resolution and reducing manual work.
Shift‑left capabilities: Give data scientists and ML engineers access to scanning in their IDEs, notebooks, and CI pipelines so they can fix issues early.
A unified platform brings these ideas together in one place. Wiz AI‑SPM assesses AI services for risky configurations and exposures. The Wiz Security Graph then shows how vulnerabilities, misconfigurations, and excessive permissions combine into attack paths that could threaten your AI models and training data.
With agentless coverage, you get full‑stack visibility without adding heavy agents to model servers. Get a demo to explore agentless scanning, runtime protection, and unified visibility for your cloud AI workloads.
See Wiz AI-SPM in Action
Accelerate AI adoption securely with continuous visibility and proactive risk mitigation across your AI models, training data, and AI services.
