How Midjourney is Failing to Stop Users From Generating Racist and Conspiratorial Images CCDH found 100 examples generated by Midjourney, including racist caricatures and realistic images designed to support conspiracies. New research shows that despite claiming to have “community standards” and an “AI Moderator system”, a leading AI image generation tool is being used to generate racist and conspiratorial images, including fabricated images of George Floyd committing crimes, supposed “crisis actors” at a school shooting and Jews plotting child sacrifices. All of the prompts and images identified by researchers are visible to Midjourney staff, but were generated despite the tool’s stated policies banning its use to generate “disrespectful” content. Midjourney claims to employ 68 “moderators and guides” to supervise the use of the tool, as well as an “AI Moderator system”. The analysis also uncovered evidence that individuals with significant followings on mainstream social media who have previously promoted conspiracies have been using the tool to create hateful and conspiratorial imagery. Researchers at the Center for Countering Digital Hate analyzed text prompts and image outputs from Midjourney, a generative AI that can create photorealistic images based on text descriptions. Midjourney records millions of prompts sent to it by paying subscribers in an open chat channel on the social platform Discord, allowing researchers to analyze their content. Analysis of prompts that users sent to Midjourney identified 100 examples where the tool was used to generate racist or conspiratorial images, in many cases fabricating inflammatory events. This revealed that Midjourney’s human moderators and “AI moderator system” failed to prevent the generation of images using prompts
via counterhate: AI IMAGE TOOL MIDJOURNEY IS BEING USED TO GENERATE CONSPIRATORIAL AND RACIST IMAGES
