Artificial intelligence is profoundly transforming cybersecurity work. No debate.

In SOCs, it analyzes huge volumes of logs and helps triage alerts. In EDR and XDR platforms, it speeds up detection and investigation. In the cloud, it spots abnormal configurations. In development, it generates and analyzes code. In risk management, it cross-references large amounts of information and helps set priorities.

The time savings can be considerable. But the more we automate, the more one question matters: how much can we trust a decision made by an AI system?

The real risk: automating judgment

The most immediate danger is not that an AI will "take control." It is that we place too much trust in it.

This phenomenon has a name: automation bias. When an answer arrives quickly and looks structured and convincing, we tend to scrutinize it less. The risk then lies as much in the system's limitations as in the trust we place in it. And an AI answer can be wrong, incomplete or based on flawed assumptions, while being delivered with confidence.

In cybersecurity, the consequences are very real. An AI can treat malicious activity as legitimate, trigger a response based on a false positive, generate vulnerable code, recommend the wrong firewall rule, underestimate a risk or expose sensitive data.

Imagine an agent in charge of handling alerts, authorized to isolate suspicious machines. One false positive on a production server, and part of the business grinds to a halt, without any attacker being involved.

So the problem is not only the quality of the model. It is also the level of authority we choose to give it.

An AI that advises is not an AI that acts

Asking an AI to "analyze this alert and suggest an action" is very different from giving it the ability to decide and execute.

In the first case, the AI remains a decision-support tool. In the second, it becomes an active component of the organization's IT systems. It must then be governed, secured, controlled and audited like any other critical component.

For sensitive actions, the preferred model is:

AI → recommendation → review → human approval → action

rather than:

AI → automatic action

That approval must also be meaningful: the person approving must have the supporting evidence, the time and the skills required, and be able to reject the action. Otherwise, automation bias takes over again.

Full automation has its place for low-impact, reversible and well-governed actions. But the greater the potential impact, the stronger the controls must be.

Applying least privilege to AI agents

A user or a system should only have the privileges strictly necessary. There is no reason for AI to be exempt from this principle.

An agent that analyzes logs does not need administrator access. An agent that queries a document repository does not need access to the entire information system. And an agent that proposes a configuration change should not necessarily be able to apply it itself.

In practice: limit access and available APIs, segment permissions, isolate environments, restrict authorized actions and require approval for sensitive operations.

An AI that is compromised, manipulated or simply wrong should run into the same barriers as a compromised user.

Controlling the data entrusted to models

Many organizations already use AI to summarize reports, analyze documents or write code. One question immediately arises: what data can we actually entrust to it?

Customer and personal data, API secrets, passwords, source code, regulated or strategic information: this information must be identified and protected.

This requires a clear policy: approved tools, data classification, contractual commitments from providers on data use and retention, and technical controls to limit leaks. Without such a policy, AI adoption is driven by individual initiatives, with no overarching governance.

A recommendation must be verifiable and traceable

In cybersecurity, an AI should not be treated as a primary source. When it recommends an action, the analyst must be able to trace that recommendation back to the supporting evidence: logs, EDR events, network flows, indicators of compromise, user context.

If the AI claims a workstation is compromised, we need to understand why. If it rates a risk as critical, we need to know the factors behind that assessment. The goal is not just a high-performing AI, but an AI whose conclusions can be challenged.

Traceability becomes just as essential. During an investigation or an audit, we will need to answer simple questions: which model and version produced this recommendation? What data was it based on, and what instruction was it given? What action was taken, and who approved it?

Interactions with AI systems must therefore be integrated into logging and monitoring, while limiting the sensitive data recorded and controlling access and retention.

AI is also an attack surface

AI is not just a defensive tool. It is itself a new attack surface: prompt injection, data poisoning, information exfiltration, agent hijacking, or abuse of the tools and APIs it is connected to.

These scenarios must be part of threat modeling. An agent connected to the information system must be tested like any other critical component.

The frameworks already exist

We do not have to build this governance from scratch.

ISO/IEC 42001 specifies the requirements for an artificial intelligence management system: responsibilities, risk management and oversight of AI use. ISO/IEC 23894 provides guidance on AI risk management. ISO/IEC 27001 remains the reference standard for information security management systems.

The core of the NIST AI Risk Management Framework is built on four complementary functions (Govern, Map, Measure, Manage): governance, context and risk identification, risk assessment and risk treatment.

On the technical side, MITRE ATLAS documents adversary tactics and techniques involving AI, based on observed attacks and realistic demonstrations. The OWASP Top 10 for LLM Applications highlights concrete risks such as prompt injection, sensitive information disclosure and excessive agency granted to agents (functionality, permissions, autonomy).

So the point is not to invent an entirely new governance model, but to extend familiar principles: risk management, least privilege, separation of duties, traceability, human oversight and continuous improvement.

The Zero Trust mindset, applied to AI

Zero Trust rests on a simple idea: never trust by default, always verify. We apply it to access by users, devices and applications. It makes sense to extend that mindset to AI: verify not only what it can access, but also what supports its recommendations.

Don't believe an answer just because it comes from an advanced model. Don't grant an agent excessive privileges just because it enables more automation. Don't confuse speed, confidence and reliability.

A human and organizational issue

AI can become a powerful force multiplier for cybersecurity teams. But a multiplier works both ways: a good decision gets executed faster, and so does a bad one.

So the question is no longer just: "Does our AI work correctly?"

But also: "Is it governed, secured, audited and controlled well enough for us to rely on its recommendations?"

The challenge is not only technological. It also involves people, organizations and decision-making.

AI should help us make better decisions. Not relieve us of the need to understand them.

What about you: how much autonomy are you willing to give AI in your security operations?