Cybersecurity experts are divided over whether hacking AI defenses can outpace hackers as criminals weaponize open-source LLMs to exploit weaknesses, as revealed in Black Hat/DEF CON briefing, to reconstruct the foundations of digital warfare.
In the scorching heat of Neveda deserts, at hacker convention DEF CON, researchers warned against AI, such as Anthropic’s Claude, could soon match senior analysts, while Palo Alto Networks predicted AI agents may be hijacked for attacks.
Meanwhile, Microsoft’s prototype detects only 24% of AI breaches, further demonstrating the intensity of defenders’ uphill battle.
Disrupting Malicious Uses of AI
Some experts argue that cybercriminals lack funding, computing power, and sophisticated tools to fully weaponize AI giving defenders valuable time to develop their own AI-powered cyberattacks skillsets.
Others warn that malicious actors are already using open-source large language models (LLMs) to scan internet-connected devices, hunt zero-day vulnerabilities, and generate malware, with their AI and cyberattacks capabilities improving rapidly.
“Cybercriminals are only going to get better, and quickly,” security researchers said at closed-door sessions during last week’s Black Hat and DEF CON conferences.
CTO of Palo Alto Networks’ Unit 42, Michael Sikorski, predicted that hackers with an AI-powered malware detection skills could soon hijack a victim’s own AI agents to launch attacks from within their systems. Although current models struggle with nuanced human-like judgments, rapid improvement could make this a near-term reality.
At DEF CON, Anthropic’s red team warned that its Claude AI model may “soon” perform at the level of a senior security researcher, potentially reducing the human workforce defending against AI-hacking tools and powered attacks.
Cybersecurity Methods for Understanding and Reducing Social Engineering Attacks
The industry’s defensive push was on display at Black Hat, where several companies showed AI-powered security solutions. Microsoft showcased a prototype agent capable of automatically detecting AI-driven cyber attacks, though its current detection rate is just 24%.
AI data breach examples such as Trend Micro introduced “digital twin” technology to safely simulate real-world cyber threats, while other firms released open-source tools for automated vulnerability detection and patching under the US-backed AI Cyber Challenge.
But these advances come with a warning— train AI to hack are adapting the same capabilities for tailored attacks. “The net effect is everybody becomes patient zero,” said John Watters, CEO of iCounter and a former Mandiant executive. Instead of reusing known exploits across multiple targets, hackers can now craft unique attack vectors for each victim, making detection and defense far more challenging.
Open-source AI has further lowered the barrier to entry, allowing attackers to run sophisticated models offline and without internet dependency, according to principal research engineer at Dreadnode, Shane Caldwell. Reinforcement learning techniques are also enabling models to evolve through malicious use of AI trial and error, reducing the need for costly supervised training.
Watters predicts a major shift by next year, with targeted cyberattacks accelerating to the point where “incident response teams are going, ‘We don’t know, we’ve never seen that before.’”
The growing arms race between hacking AI defenders and adversaries underscores a central reality of modern cybersecurity AI is both the newest shield and the sharpest sword.
Inside Telecom provides you with an extensive list of content covering all aspects of the tech industry. Keep an eye on our Cybersecurity sections to stay informed and up-to-date with our daily articles.