Extremist movements are seeking voice cloning AI technology to fuel their propaganda machine, by copying sounds, spreading messages, and recruiting supporters, changing the conception of global security risks and the flow of online information.
Human effort is no longer seen where everything has become software-based and can recreate speech in minutes and imitate voices, lowering barriers for extremist groups, allowing ideas to travel faster, wider, and with big emotional impact without clear oversight.
AI Cloning Voice for Extremism
AI is no longer limited to creative and commercial purposes, and voice cloning AI is reviving historical speeches and delivering content that grabs the targeted people’s attention and sentiment.
AI propaganda is reaching new audiences who may never read written proposals.
“The adoption of AI-enabled translation by terrorists and extremists marks a significant evolution in digital propaganda strategies,” said Lucas Webber, a senior threat intelligence analyst at Tech Against Terrorism and a research fellow at the Soufan Center.
Voice cloning AI is used by neo-Nazi far right groups, to generate English versions of Hitler’s speeches, grabbing the attention of viewers on multiple platforms like X (FKA Twitter), TikTok, and Instagram.
A Global Network on Extremism and Technology (GNet) research revealed that some content creators rely on the best voice cloning tools to give the generated content a realistic pattern.
Groups linked to violent ideologies have also transformed written material into audio. As an example, Siege, an insurgency text written by American neo-Nazi was published as an audiobook using AI Cloning Voice technology.
“For the last several months I have been involved in making an audiobook of Siege by James Mason,” said a prominent neo-Nazi influencer.
Weak Safeguards Against AI Voice Cloning Scam
Misusing AI has become the foundation behind voice cloning scams that are fueling political manipulation.
According to a Consumer Reports investigation, most services lack strong barriers, making the abuse of AI impersonation voice easier.
Voice cloning systems require only seconds of audio and sometimes are pulled from social media. This initiated AI voice impersonation scams, where offenders mimic public figures. This has led regulators to respond to some cases that involve fake political content; however, law enforcement remains limited.
“The creator of the audiobook has previously released similar AI content; however, Siege has a more notorious history,” said Joshua Fisher-Birch, a terrorism analyst at the Counter Extremism Project.
In parallel, there’s also the Jihadist groups that also adopt voice cloning AI technology to turn text into speech and expand multilingual outreach.
According to Webber, extremist State outlets are “using AI to create text-to-speech renditions of ideological content from official publications” to boost engagement.
Simultaneously, some users openly discuss how AI impersonating voices make translation easier. But this does not blur the fact that weak regulations and controls play a huge role in spreading voice cloning scams and its tactics.
Platforms usually seek self-policing, even as open source voice cloning software becomes more accessible. Sarah Myers West, co-executive director of the AI Now Institute, says that some AI voice cloning is legitimate in some cases, for the purpose of helping people with disabilities and creating audio translations in different languages, however, she does not ignore the harm it causes.
As authorities try to catch up with AI and technology, extremists and criminals continue to test what many still call the best voice cloning options. Raising fears that unchecked tools may normalize an AI impersonal voice that is hard to trace or trust. The issue is deep, and the future is scary.
Inside Telecom provides you with an extensive list of content covering all aspects of the tech industry. Keep an eye on our Ethical Tech section to stay informed and up-to-date with our daily articles.