Filigran warns AI security needs tighter human oversight

Filigran marked this year's AI Appreciation Day by warning that rapid advances in artificial intelligence for cybersecurity require tighter human oversight and more deliberate deployment. The open-source threat management specialist shared new commentary on how agentic AI is reshaping defensive operations and risk management.

Observed each year on 16 July, AI Appreciation Day highlights both the benefits and risks of AI. Filigran executives used the occasion to show how automated agents are moving beyond analysis into orchestrated response across complex technology environments.

Demand for automation in exposure management is rising, according to Filigran. Its State of Threat Management report found that 37% of exposure management processes are AI-driven, with respondents expecting that figure to reach 59% within two years.

Agentic AI features strongly in the company's outlook. Executives pointed to a shift from tools that only surface alerts to systems that correlate threat intelligence, validate real exposure, and trigger remediation workflows under human supervision.

Neena Sharma, Head of Customer and Product Marketing at Filigran, said AI is beginning to close the long-standing gap between detection and action in security operations.

"In the realm of security, AI is changing proactive threat management. With AI, threat management stops being a slide in a deck and starts being a practice. AI is not only speeding things up, it is gaining the ability to close gaps between detection and action. I'm especially excited by  agentic AI where AI can execute as an orchestrator: correlating intel, validating exposure and kicking off remediation workflows autonomously. With a  human-in-the-loop, of-course," said Neena Sharma, Head of Customer and Product Marketing at Filigran.

Front-line specialists at Filigran reported similar trends inside security operations centres. They said autonomous agents now cross-reference indicators against live environments, filter noise, and carry out containment steps at speed.

"AI's biggest impact is that it's rapidly closing the gap between spotting a threat and neutralizing it. Rather than analysts manually sifting through thousands of alerts, autonomous agents can now cross-reference indicators against an organization's real environment, filter out the noise, test actual exposure, and trigger containment in real time. The result: security teams that move from reactive triage to genuine proactive management, catching and addressing threats before they escalate," said Deborah Galea, Cybersecurity Specialist at Filigran.

"Modern defense models track how users and devices actually behave, so attackers hiding behind stolen credentials don't stay hidden for long. Feed enough alerts into the right automated workflows and the system can isolate a compromised endpoint or kill leaked access in milliseconds - faster than any analyst could react manually. Add dark web monitoring and regular breach simulations on top, and risk management stops being something you do after the fact," said Falk Schwendike, Senior Solutions Engineer at Filigran.

Despite that momentum, Filigran leaders said the next phase of AI remains uncertain and could prove destabilising if organisations expand automation without clear guardrails.

"The way we are seeing Frontier AI making advancement, it can be difficult to predict how AI will evolve over the next year. . In the present, we are already seeing how vulnerability discovery time is shrinking. However, we need to be careful about blind uptake of these tools as it's a double-edged sword. The winners won't be who adopts AI fastest, it'll be who adopts it deliberately. Security teams must focus on how they  want to be able to utilize AI to improve defenses, not to open the attack surface even wider," said Sharma.

"By next year's AI Appreciation Day, I expect the industry to have moved further toward autonomous defensive agents operating at machine speed. But that progress only counts if it's built on strict architectural guardrails, not left to the AI's own judgment. Without those boundaries, the very agents meant to defend us risk becoming threats themselves," said Galea.

Several Filigran experts expect an AI-driven arms race between attackers and defenders, with both sides relying more heavily on automation and machine-speed decision-making around vulnerabilities and exploits.

"By next AI Appreciation Day, cybersecurity will be defined by an AI vs. AI arms race. Bad actors will be able to autonomously launch attacks and security teams will defend against them, all in real time. The window between vulnerability disclosure and weaponization will shrink to machine-like speed. But, state and commercial cyber threat intelligence (CTI) teams use AI to discover zero-days and predict Indicators of Compromise (IoCs) before attackers strike," said Jake Taylor, Head of Government, NEMEA, at Filigran.

"Looking ahead, we will see zero-human remediation taking over, with systems rewriting their own firewall rules and spinning up clean mirror environments to keep businesses running before an analyst is even paged. For penetration testing, autonomous agent swarms will hunt for vulnerabilities around the clock, quietly writing and deploying their own micro-patches for zero-day exploits before vendors even realize they exist. On the privacy front, enterprises will completely walk away from public AI APIs, choosing to run highly compressed, air-gapped language models on their own hardware so every threat prediction stays strictly inside the building. Finally, supply chain auditing will achieve true machine speed, meaning every single piece of third-party code is inspected and cleared at compilation, while auditable compliance reports assemble themselves the moment they are requested," said Schwendike.

Filigran executives also outlined guidance on supervising AI systems, including stricter data controls, clearer permissions, and a shift away from informal prompt writing toward structured workflows.

"Security analysts should treat AI like a very fast but junior teammate: useful, but not yet accountable. We need to guide it with the same rigor you'd apply to any new hire: clear guardrails, human-in-the-loop and more autonomy earned over time. That means clear policies on what AI tools can touch, what data they can access, and what actions they're allowed to take autonomously versus what needs sign-off," said Sharma.

"Security teams need to shift from writing simple prompts to acting as AI governors and verifiers. That means setting clear boundaries, not just trusting judgment: an agent should never be able to make a call like isolating a core database without human approval. And as teams hand agents more autonomy, they have to treat securing those agents themselves as a top priority, not an afterthought," said Galea.

"For AI-powered security tools to actually take work off people's plates instead of burying them in false alarms, security teams need to steer the technology rather than trust it blindly. Look far enough ahead, and four areas stand out. First, there's context. Teams will have taught their AI that a break-in on an intern's laptop is a different animal entirely from someone touching the customer database, and clean exception lists will mean the system stops flinching every time IT runs its nightly backup. Second, humans stay in the loop. The big calls-locking an executive's account, pulling the plug on a production line-still wait for a person to click approve. And when the AI gets something wrong, analysts won't just dismiss it; they'll correct it, so the system actually learns. Third, prompt engineering becomes routine. Teams build up libraries of tried-and-tested threat-hunting queries, and when they lean on an AI assistant, they give it a real job to do - something like "act as a seasoned incident response analyst and check this script for obfuscation." Fourth, the AI itself gets locked down. Nobody wants source code or internal logs leaking into a public model, so that gets watched closely - and the training data behind in-house systems gets protected too, so no one can quietly poison the AI's judgment from the outside," said Schwendike.