Thought Leadership
Deepfakes, compromised AI applications, prompt injection, and software supply chain attacks are the four security threats where defenders are currently at a clear disadvantage, according to new analysis from Gartner. The rise of generative AI is not easing the pressure — in fact, it is amplifying it.
The findings were presented at Gartner’s Security & Risk Management Summit in National Harbor, USA. The research firm highlights four threat areas where attackers are gaining the upper hand: deepfakes, AI application compromise, prompt injection, and software supply chain attacks.
Gartner maps these risks in a “ThreatScape” model, which evaluates threats along two dimensions: the availability of reliable threat intelligence signals and the level of organizational readiness to mitigate them. Ironically, security initiatives from leading AI vendors are adding noise to an already complex landscape.
“Security initiatives from frontier AI providers are creating significant noise in an already noisy threat environment,” said analyst John Watts. Security teams, he added, must “find the signal in all the noise” to respond effectively to shifting risks.
Compromised AI Applications
As AI tools become more widely deployed in production environments, their attack surface expands significantly. This no longer applies only to public-facing services, but also to internally developed agents, third-party integrations, and employee-facing applications.
Weak controls can expose sensitive data or credentials. “Security teams must extend their programs beyond traditional software protection,” Watts said.
This includes systematically mapping the attack surface created by GenAI models and agentic tools. Gartner’s own TRiSM framework (Trust, Risk and Security Management) is designed to support this effort.
Specifically, the company recommends applying secure development lifecycles and threat modeling to AI systems as well, strengthening data security through improved classification, implementing purpose-based access control (PBAC), and monitoring runtime behavior. Organizations do not need to start from scratch, as a growing number of startups already offer relevant tools on the market.
Deepfake-Driven Identity Fraud
Generative AI has dramatically increased the scale, realism, and accessibility of deepfakes across voice, video, and image. These can be used both as pre-generated content and in real-time attacks.
Attackers are leveraging deepfakes to bypass biometric authentication, execute real-time social engineering attacks, and even infiltrate hiring processes. According to Watts, deepfakes are now “commonplace” in fraud and phishing campaigns designed to evade detection.
No single control is sufficient. “There is no single security check that will protect you,” Watts warned. Instead, organizations need layered defenses combining hardened processes, employee awareness, and detection technologies.
For biometric verification systems, Gartner recommends focusing on presentation attack detection and injection attack detection, supported by contextual signals. Online meetings can be protected through conditional access policies and metadata analysis.
Software Supply Chain Attacks
“The evolution of GenAI services will only accelerate the trend toward software supply chain attacks exploiting vulnerabilities in open-source software,” Watts said.
Organizations are urged to shift toward trusted component registries, strengthen CI/CD pipelines, and improve anomaly detection and response capabilities.
Gartner recommends requiring SBOMs (Software Bill of Materials) and AIBOMs from vendors, verifying components against current threat intelligence, and restricting third-party code, containers, and AI models to curated sources. Code repositories should be protected with branch controls, while artifacts must be signed at build time. Build systems should follow least-privilege principles, and runtime behavior of agent-based systems must be continuously monitored.
Prompt Injection Attacks
The fourth major threat identified is prompt injection, where attackers manipulate large language models (LLMs) through crafted inputs. These attacks can force models to leak sensitive information, perform unauthorized actions, or bypass safety controls. As generative AI adoption grows, so does the attack surface.
Gartner recommends a multi-layered defense strategy: input validation and sanitization, continuous monitoring for anomalous model behavior, and integrating prompt injection testing directly into the development lifecycle. Results should feed back into runtime protection mechanisms.
