Does your penetration test only examine the web layer, or also what your AI assistant or agent does with tools and data?
An LLM penetration test examines chatbots, copilots, and AI agents to determine whether attackers can redirect them via manipulated input. The focus is on direct and indirect prompt injection, disclosure of system context, and the misuse of connected tools. Permissions in connected knowledge bases (RAGs) are also assessed. Testing is conducted according to the OWASP Top 10 for LLM Applications, the OWASP API Security Top 10, the OWASP Web Security Testing Guide (WSTG) 4.2, and the BSI/ACS guideline for penetration testing of large language models (LLMs).
An application with a language model processes inputs that you cannot trust and makes decisions or calls as AI agent independently connected tools. This creates an attack surface that is classic Web exams Do not cover: The boundary between instruction and data becomes blurred, and a cleverly worded text can change the behavior of the system.
Practically relevant attacks are no longer a game. Indirect prompt injection via an embedded document, intercepting system context, misusing connected tools, or exceeding permissions through model access are all real methods. We examine the application as a whole, the model, its context, the connected data sources and tools, according to established references for AI security.
We are checking whether the behavior can be controlled via input and whether embedded content from documents or web sources is injecting unwanted instructions.
We are testing whether system instructions, hidden rules, or sensitive context data can be extracted from the model.
Where the model triggers tools or actions, we check whether these can be misused via manipulated inputs.
We are examining whether the approach via this model circumvents permissions and opens access to data that users are not entitled to, such as in connected knowledge sources.
We assess whether unsafe or harmful expenditures are being incurred and could cause damage in downstream systems.
An LLM penetration test attacks your AI application in the same way an attacker would via input, documents, or web content, from user input to connected data sources and tools. It is designed for companies that use or offer chatbots, copilots, RAG applications, or AI agents. Whether the test is worthwhile depends primarily on the model's permissions. If it accesses internal knowledge, retrieves data, executes tools or code, or triggers processes, we examine whether this could be exploited for data leaks, sandbox escapes, or workflow abuse. Because the AI layer builds upon this, combining it with web applications and APIs or the cloud is often beneficial. Following this, a technical workshop is recommended; for regulatory requirements, we refer you to ProSec Governance.
We clarify the architecture, connected data sources and tools, roles and test parameters, including a frozen state.
We derive the relevant attack scenarios from the specific application, instead of testing generically.
We systematically work through the scenarios and document attack success and, where applicable, detection.
Findings and assessments are cross-checked before handover.
You will receive findings including the route of attack, impact, and recommendation, and you will review them with us.
First, the technical findings for your IT team, then the remediation plan, then the summary for your management, and finally the audit report for customers, auditors, or insurers. Everything is based on the same findings.
Each finding includes documentation, classification according to severity, impact, and relevance to your environment, along with a list of affected systems and devices. Transparent and, upon request, available in a [document/format - context needed]. Retest verifiable.
A prioritized action plan with clear responsibilities, optionally refined in a technical workshop with your team. It forms the basis for resolving the issue, and upon request, we will support the implementation through to retesting.
Situation, risk, and decision-making requirements summarized in just a few pages, easily understandable without technical expertise. This allows you to engage management, committees, and specialist departments, communicate the findings, and identify necessary actions.
Confirmation of the completed audit, without disclosing technical details. For clients, auditors, regulators, and your cyber insurance.
This analysis examines the AI layer, including its model, context, and connectivity. Beneath this lie the web application, API, and cloud environment, and with ongoing development, the attack surface changes with each release.
When the AI function is embedded in an existing web application or API, Web Applications & APIs checks authentication, sessions, and underlying business logic.
Regarding the performanceCan be combined as soon as the model, knowledge sources or tools run in the cloud or in Microsoft 365 and their identities and permissions are to be evaluated.
Regarding the performanceWhen prompts, models, or connections change regularly, checks performed at the release frequency keep the status up to date.
Regarding the performanceBriefly describe what it's about. We'll work out the details with you during the scoping process. For the consultation, we need...
You don't need to prepare; we'll guide you expertly step by step. Based on your business processes and IT systems, we'll work together to determine the right testing concept.
These are the questions we most frequently receive regarding this service. We'd be happy to clarify anything that's missing in a personal consultation.
An LLM penetration test can provide a building block for the requirements of the AI Act, but it does not replace a conformity assessment. For high-risk AI systems, the AI Act requires, among other things, an appropriate level of accuracy, robustness, and cybersecurity (Article 15 AI Act). Since the amendment by Regulation (EU) 2026/1744, the obligations for high-risk AI systems according to Annex III apply from December 2, 2027, and those according to Annex I from August 2, 2028. Regulatory support is provided by ProSec Governance.
Both, depending on the question. The most informative approach is to test the application as a whole: the model in conjunction with your data sources, tools, and permissions. We also offer targeted tests for just the model or just your agents. We determine the most suitable approach during the scoping phase.
We work with a defined, frozen state during testing and document the attack paths in such a way that they remain traceable and repeatable. A retest is advisable if there is a significant change to the model or the connection.
An LLM penetration test, also known as an AI penetration test, examines a language model application for attacks against the interplay of input, model, context, and connected tools. These attacks include prompt injection, disclosure of system instructions, and abuse of privileges. References include the OWASP Top 10 for LLM Applications, the OWASP API Security Top 10, and OWASP WSTG 4.2.
In direct prompt injection, the manipulative instruction is embedded in the user's input, while in indirect prompt injection, it is embedded in content processed by the model, such as a document, email, or website. The indirect variant is often more dangerous because the attacker does not need direct access to the application. We are investigating both methods.
A test that provides insight into system instructions, connections, and permissions will be significantly more thorough in the same amount of time. A test without prior knowledge is also possible and shows what an external attacker would see. We determine the appropriate scope during the scoping process.
Yes, within the limits of what the provider allows. Your configuration and its connection to your own data or tools will be tested, not the provider's base model. Methodologically, this is closer to an audit, using penetration testing techniques. Testing a Copilot without your own integrations alone is not very useful. For Microsoft 365 Copilot, testing under Cloud & Microsoft 365 is usually the more sensible approach. This testing is similar to an audit but uses penetration testing methods. Testing a Copilot without integrations is not very useful; then a Cloud audit The more sensible approach. If it's connected to data, tools, and workflows, it's well worth examining.
The LLM penetration test specifically checks your application for vulnerabilities, AI Red Teaming This often additionally includes testing for harmful or undesirable model outputs. The terms are not used consistently in the market. Our focus is on application security: prompt injection, data leakage, tool misuse, and permissions. We also assess unsafe outputs insofar as they endanger downstream systems.
Yes, and this is where the audit is particularly important because a redirected model doesn't just respond, it acts. We examine which tools, interfaces, and permissions an agent can use and whether these can be misused via manipulated content. In the OWASP Top 10 for LLM Applications, this risk is called Excessive Agency. We also audit MCP servers, through which agents access tools, preferably by examining the source code.
It depends primarily on the number of connected tools, data sources, and roles, and on whether we gain insight into system instructions and architecture. A chatbot without any connections can be tested more quickly than an agent that sends emails or creates tickets. Because model responses vary, we repeat attacks multiple times, and this also factors into the effort.
It depends on what the LLM is allowed to do. A simple chatbot on the website that can't execute anything is hardly worth the effort. If the model is more deeply integrated into your web application or API, or connected to internal knowledge, tools, and workflows, then testing is highly worthwhile. We examine precisely these areas to determine whether internal knowledge can be extracted, whether code can be executed via the connected tools, whether it's possible to escape the sandbox, and whether workflows can be exploited for attacks.
Yes. MCP servers connect models with tools and data and are therefore part of the attack surface. We prefer to examine them using source code, especially if they are custom-developed: which tools they provide, how they authenticate, and what a manipulated call can trigger.
