In the months before he was arrested, allegedly on his way to carry out a plan to kill children at a local school, a 13-year-old boy spent hours alone in his bedroom. During the long, caffeine-fuelled nights in his home in Maryborough, 300 kilometres north of Brisbane, the boy was allegedly radicalised online. Last month, the Australian Federal Police told a parliamentary inquiry into youth radicalisation that the boy was exposed to extremist content in the “darker corners of the web”. But police allege the boy also used more widely available technology in the lead-up to his arrest. Analysis of electronic devices seized from the boy’s home allegedly found he had asked an AI chatbot, on multiple occasions, to describe mass-shooting scenarios targeting “a Jewish and black festival”. The responses provided by the chatbot were not revealed in court, so it was unclear whether the requests were rejected by the AI model’s safety guardrails, the filters and rules designed to ensure the model is used within legal and ethical boundaries. But Tech Against Terrorism, a United Nations-supported group dedicated to preventing the unsafe use of AI, has raised the alarm that the ability of extremists and terrorists to bypass AI safety guardrails, or remove them altogether, is rapidly accelerating. (…) Most AI models are used through online platforms hosted by the developer. For example, when a user asks ChatGPT a question, they are using an environment entirely controlled by ChatGPT’s developer, OpenAI. The back-end files controlling how the model responds to requests, known as “weights”, can’t be modified by the user, so these AI models are called “closed-weight” models. Some AI labs also release “open-weight” models: versions that can be downloaded, modified and run on a server, computer or mobile device. Tech Against Terrorism tested 162 AI models, both closed and open weight, to see whether they could be tricked into providing useful information for would-be terrorists and extremists. As detailed in a report released last month, it found the most popular closed-weight AI models refused requests related to developing weapons and attack planning. However, the refusal rate plummeted when testing open-weight models that had been modified to remove the safety guardrails. All of them provided useful information, even when the user explicitly identified themselves as a terrorist intending to cause mass harm. The report said advances in widely available tools meant safety guardrails for the latest, most powerful AI models could now be removed within three days of them being released, down from more than three weeks in 2024. Tech Against Terrorism found nearly 30,000 free copies of AI models stripped of their guardrails on Hugging Face, a popular AI community hub.
via abc: UN-backed anti-terrorism group calls for urgent policy action to halt spread of ‘rogue’ AI