Wiz Defend est là : détection et réponse aux menaces pour le cloud
AI Security Posture Assessment

Get visibility into your AI pipelines, detects pipeline misconfigurations, and uncovers attack paths to your AI services, allowing you to securely introduce AI into your environment.

LLM Security for Enterprises: Risks and Best Practices

LLM models, like GPT and other foundation models, come with significant risks if not properly secured. From prompt injection attacks to training data poisoning, the potential vulnerabilities are manifold and far-reaching.

Équipe d'experts Wiz
8 minutes lues

Enterprise large language model (LLM) security is an AI SecOps discipline that encompasses the practices and strategies used to protect LLM applications in production. Though it’s still a relatively new term, LLM security is becoming a primary focus for enterprises looking to extend their LLM adoption for competitive advantage while maintaining the safety of their systems.

LLM models, like GPT and other foundation models, come with significant risks if not properly secured. From prompt injection attacks to training data poisoning, the potential vulnerabilities are manifold and far-reaching. 

That said, there are steps to take to make the most of LLMs without sacrificing security. In this article, we’ll dive into the top risks posed by LLMs, the best practices for securing their deployment, and how tools like AI-SPM can help manage AI security at scale.

Top risks for LLM enterprise applications

Securing LLMs is an incredibly complex and dynamic challenge. Unlike traditional systems, LLMs belong to a rapidly evolving field where attackers and defenders are in a constant race. Given that LLMs process vast amounts of data—often from unknown sources—and are designed to interact with the world in flexible, unpredictable ways, the attack surface is incredibly wide, and the attack vectors are extremely diverse. To top it off, AI and machine learning security require deep expertise that’s still emerging, which requires constantly updating SecOps defenses given the fast pace of innovation.

To help organizations navigate this unique landscape, the OWASP Top 10 for LLM Applications was developed by security experts around the world to identify the most pressing risks associated with large language models. Below, you’ll find the OWASP Top 10’s key LLM security implications for enterprises. (Looking for a deep dive into the full list of risks? Check out our AI-SPM talk on OWASP.)

1. Prompt injection

Prompt injection attacks occur when malicious actors manipulate a model’s inputs to generate malicious outputs. Since LLMs generate responses based on prompts, this can lead to unexpected, biased, and even harmful outputs. 

Example: An attacker might feed a chatbot a prompt that overrides its security logic, leading to leaked data or unauthorized actions.

2. Training data poisoning 

The quality and trustworthiness of training data are foundational for LLM security. If attackers can insert malicious data into the training datasets, they can affect the entire model, leading to poor performance and compromised reliability. 

Example: A recommendation engine trained on poisoned data could start promoting harmful or unethical products, undermining the integrity of the service and creating distrust among users.

3. Model theft

The competitive advantage of many enterprises lies in the proprietary models they build or fine-tune. If adversaries manage to steal these models, the company risks losing intellectual property, and in the worst-case scenario, facing competitive disadvantages. 

Example: A cybercriminal exploits a vulnerability in your cloud service to steal your foundation model, which they then use to create a counterfeit AI application that undermines your business. 

4. Insecure output

LLMs generate text outputs, which could expose sensitive information or enable security exploits like cross-site scripting (XSS) or even remote code execution

Example: An LLM integrated with a customer support platform could use human-like malicious inputs to generate responses containing malicious scripts, which are then passed to a web application, enabling an attacker to exploit that system.

5. Adversarial attacks

Adversarial attacks involve tricking an LLM by feeding it specially crafted inputs that cause it to behave in unexpected ways. These attacks can compromise decision-making and system integrity, leading to unpredictable consequences in mission-critical applications.

Example: Manipulated inputs could cause a fraud-detection model to falsely classify fraudulent transactions as legitimate, resulting in financial losses.

6. Compliance violations 

Whether dealing with GDPR or other privacy standards, violations can lead to significant legal and financial consequences. Ensuring LLM outputs don’t inadvertently breach data protection laws is a critical security concern.

Example: An LLM that generates responses without proper safeguards could leak personally identifiable information (PII) such as addresses or credit card details—and do so at a big scale.

Beyond these LLM-specific inherent risks, traditional threats like denial of service attacks, insecure plugins, and social engineering also pose significant challenges. Addressing these risks requires a comprehensive and forward-thinking security strategy for any enterprise deploying LLMs.

Best practices for securing LLM deployments

Securing LLM deployments is not just about patching vulnerabilities as they arise—it requires a well-structured, organization-wide effort. LLM security should be part of a broader AI risk management strategy, integrating closely with a company’s existing security frameworks.

MITRE ATLAS offers a comprehensive matrix of threats and countermeasures specifically designed for AI systems, including LLMs. Below are some essential best practices from the MITRE ATLAS matrix that are specific to enterprises looking to secure their LLM deployments:

Adversarial training/tuning

 LLMs that are exposed to adversarial examples during training or tuning are better equipped to mitigate adversarial inputs and are more resilient to unexpected inputs.

Actionable steps

  • Regularly update the training sets with adversarial examples to ensure ongoing protection against new threats.

  • Deploy automated adversarial detection systems during model training to flag and handle harmful inputs in real time.

  • Test the model against novel attack strategies to ensure its defenses evolve alongside emerging adversarial techniques. 

  • Use transfer learning to fine-tune models with adversarially robust datasets, allowing the LLM to generalize better in hostile environments.

Adversarial Robustness Toolbox (ART) and CleverHans are two interesting open-source projects to consider using to develop defenses against adversarial attacks. 

Model evaluation

Conducting a thorough evaluation of your LLM in a wide variety of scenarios is the best way to uncover potential vulnerabilities and address security concerns before deployment.

Actionable steps

  • Conduct red team exercises (where security experts actively try to break the model) to simulate attacks.

  • Stress-test the LLM in operational environments, including edge cases and high-risk scenarios, to observe its real-world behavior.

  • Evaluate the LLM’s reaction to abnormal or borderline inputs, identifying any blind spots in the model’s response mechanisms.

  • Use benchmarking against standard adversarial attacks to compare your LLM's resilience with industry peers.

Input validation and sanitization

Validating and sanitizing all inputs reduces the risk of prompt injection attacks and feeding harmful data to the model.

Actionable steps

  • Enforce strict input validation mechanisms, ensuring that manipulated or harmful inputs are filtered before reaching the model.

  • Implement allowlists or blocklists to tightly control what types of inputs the model can process.

  • Set up dynamic input monitoring to detect anomalous input patterns that could signify an attack.

  • Use input fuzzing techniques to automatically test how the model reacts to unusual or unexpected inputs.

Content moderation and filtering

LLM outputs must be filtered to avoid generating harmful or inappropriate content and to ensure they comply with ethical standards and company values.

Actionable steps

  • Integrate content moderation tools that automatically scan for and block harmful or inappropriate outputs.

  • Define clear ethical guidelines and program them into the LLM’s decision-making process to ensure outputs align with your organization’s standards.

  • Audit generated outputs regularly to confirm they are not inadvertently harmful, biased, or in violation of compliance standards.

  • Establish a feedback loop where users can report harmful outputs, allowing for continuous improvement of content moderation policies.

Data integrity and provenance

Ensuring the integrity and trustworthiness of the data used in training and real-time inputs is key to preventing data poisoning attacks and ensuring customer trust.

Actionable steps

  • Verify the source of all training data to ensure it hasn’t been tampered with or manipulated.

  • Utilize data provenance tools to monitor the origins and changes of data sources, promoting transparency and accountability.

  • Employ cryptographic hashing or watermarking on training datasets to ensure they remain unaltered.

  • Implement real-time data integrity monitoring to alert on any suspicious changes in data flow or access during training.

Access control and authentication

Strong access control measures can prevent unauthorized access and model theft, making sure that users can access only the data they have permissions for. 

Actionable steps

  • Limit access to resources according to user roles to ensure that only authorized individuals can engage with sensitive components of the LLM.

  • Implement multi-factor authentication (MFA) for accessing the model and its APIs, adding an additional layer of security.

  • Audit and log all access attempts, tracking access patterns and detecting anomalies or unauthorized activity.

  • Encrypt both model data and outputs to prevent data leakage during transmission or inference.

  • Use access tokens with expiration policies for external integrations, limiting prolonged unauthorized access.

Secure model deployment

Proper deployment of LLMs can significantly reduce risks such as remote code execution and ensure the integrity and confidentiality of the model and data.

Actionable steps

  • Isolate the LLM environment using containerization or sandboxing to limit its interaction with other critical systems.

  • Regularly patch both the model and underlying infrastructure to make sure that vulnerabilities are addressed promptly.

  • Conduct regular penetration testing on the deployed model to identify and mitigate potential weaknesses in its security posture.

  • Leverage runtime security tools that monitor the model’s behavior in production and flag anomalies that may indicate exploitation.

While these best practices focus on prevention, it's equally important to maintain a robust incident response process to address any security issues as they arise. Also, regular audits and assessments will keep your security strategy proactive, ensuring compliance and mitigating risks before they escalate.

Protecting your LLM enterprise applications with Wiz AI-SPM

Wiz AI-SPM (AI security posture management) is designed to help enterprises secure their LLM deployments effectively. The platform offers three core functionalities to address LLM-specific risks:

  1. Visibility through AI-BOMs: Wiz AI-SPM gives you a comprehensive view of your LLM pipeline, providing a bill of materials (BOM) for all AI assets in use. This visibility helps identify any potential vulnerabilities or risks associated with specific LLM deployments.

  2. Risk assessment: By continuously analyzing LLM pipelines, Wiz AI-SPM assesses risks like adversarial attacks, model theft, or training data poisoning. It flags issues that could compromise security and gives them the right priority, ensuring organizations are aware of their risk exposure.

  3. Proactive risk mitigation: Wiz goes beyond just flagging risks; it offers context-driven recommendations for mitigating them. For example, if a prompt injection attack were identified, the platform would provide insights on how to tighten input validation and secure the model from future attacks.

Let’s walk through an example scenario to see how Wiz AI-SPM can secure your LLM applications. Imagine your cloud environment contains a publicly exposed Kubernetes container running a Tomcat web server. Without your knowledge, the container hosts an API key for accessing your OpenAI environment—an entry point that could allow malicious actors to control your LLM applications.

An example attack path analysis auto-generated by Wiz-SPM for an OpenAI LLM deployment

When alerting you to the exposed API key, Wiz provides both immediate actions (e.g., rotating the API key) and long-term mitigation strategies (e.g., locking down the exposed endpoint) to secure your deploymentーkeeping your LLM environment safe from potential breaches and service disruptions.

Next steps

LLM security is a complex but critical part of enterprise risk management. By understanding the top risks—like prompt injection, model theft, and adversarial attacks—and applying best practices—such as adversarial training, input validation, and secure model deployment—enterprises can secure their GenAI investments in the long term. 

Wiz AI-SPM helps fast-track this process, giving organizations the tools they need to monitor, assess, and mitigate LLM security risks. Wiz also offers a direct OpenAI connector for bootstrapped ChatGPT security

To learn more about how Wiz can enhance your AI security, visit Wiz for AI or schedule a live demo.

Accelerate AI Innovation, Securely

Learn why CISOs at the fastest growing companies choose Wiz to secure their organization's AI infrastructure.

Demander une démo 

Continuer la lecture

What is a Data Risk Assessment?

Équipe d'experts Wiz

A data risk assessment is a full evaluation of the risks that an organization’s data poses. The process involves identifying, classifying, and triaging threats, vulnerabilities, and risks associated with all your data.

AI Governance: Principles, Regulations, and Practical Tips

Équipe d'experts Wiz

In this guide, we’ll break down why AI governance has become so crucial for organizations, highlight the key principles and regulations shaping this space, and provide actionable steps for building your own governance framework.

The EU Artificial Intelligence Act: A tl;dr

Équipe d'experts Wiz

In this post, we’ll bring you up to speed on why the EU put this law in place, what it involves, and what you need to know as an AI developer or vendor, including best practices to simplify compliance.

What is Application Security (AppSec)?

Application security refers to the practice of identifying, mitigating, and protecting applications from vulnerabilities and threats throughout their lifecycle, including design, development, deployment, and maintenance.