Researchers warn generative tools are helping militant groups from neo-Nazis to the Islamic State spread ideology. While the artificial intelligence boom is upending sections of the music industry, voice generating bots are also becoming a boon to another unlikely corner of the internet: extremist movements that are using them to recreate the voices and speeches of major figures in their milieu, and experts say it is helping them grow. “The adoption of AI-enabled translation by terrorists and extremists marks a significant evolution in digital propaganda strategies,” said Lucas Webber, a senior threat intelligence analyst at Tech Against Terrorism and a research fellow at the Soufan Center. Webber specializes in monitoring the online tools of terrorist groups and extremists around the world. (…) On the neo-Nazi far-right, adoption of AI-voice cloning software has already become particularly prolific, with several English-language versions of Adolf Hitler’s speeches garnering tens of millions of streams across X, Instagram, TikTok, and other apps. According to a recent research post by the Global Network on Extremism and Technology (GNet), extremist content creators have turned to voice cloning services, specifically ElevenLabs, and feed them archival speeches from the era of the Third Reich, which are then processed into mimicking Hitler in English. Neo-Nazi accelerationists, the kinds who plot acts of terrorism against western governments to provoke a societal collapse, have also turned to these tools to spread more updated versions of their hyper-violent messaging. For example, Siege, an insurgency manual written by American neo-Nazi and proscribed terrorist James Mason that became the veritable bible to organizations like the Base and the now-defunct Atomwaffen Division, was transformed into an audiobook in late November. “For the last several months I have been involved in making an audiobook of Siege by James Mason,” said a prominent neo-Nazi influencer with a heavy presence on X and Telegram, who stitched together the audiobook with the help of AI tools. “Using a custom voice model of Mason, I re-created every newsletter and most of the attached newspaper clippings as in the original published newsletters.” The influencer lauded the power of having Mason’s writing from “pre-internet America” and turning it into a modern-day voice. “But to hear the startling accuracy of predictions made through the early eighties really puts a milestone on the road and it changed my view of our shared cause on a fundamental level,” he said. At its height in 2020, the Base held a book club on Siege, which was an instrumental influence on several members who discussed its benefits in a hypothetical war against the US government. A nationwide FBI counterterrorism probe eventually swept up over a dozen of its members on various terrorism related charges in the same year. “The creator of the audiobook has previously released similar AI content; however, Siege has a more notorious history,” said Joshua Fisher-Birch, a terrorism analyst at the Counter Extremism Project, “due to its cultlike status among some in the online extreme right, promotion of lone actor violence, and being required reading by several neo-Nazi groups that openly endorse terrorism and whose members have committed violent criminal acts”.
via guardian: Extremists are using AI voice cloning to supercharge propaganda. Experts say it’s helping them grow