For months before carrying out Canada’s deadliest school shooting in almost four decades, Jesse Van Rootselaar was consulting an unlikely adviser on how to perpetrate mass murder.
“In a hallway, indoors, or a crowded classroom, the ۸۷۰ [Remington shotgun] is brutal … Close quarters is its playground,” OpenAI’s ChatGPT guided the ۱۸-year-old, according to reports.
Recommended Stories
list of ۳ itemsend of list
On February ۱۰, ۲۰۲۶, Van Rootselaar shot his mother and half-brother at home before going to a school in the small British Columbia town of Tumbler Ridge, where he killed another six people – including five students – before taking his own life.
Earlier this week, on October ۱۴, US authorities charged a Washington State man with allegedly conspiring with Van Rootselaar to plan the attack. But the mass shooting incident has raised questions that go beyond the traditional legal understanding of culpability.
The British Columbia government in September filed a civil lawsuit against OpenAI and its CEO Sam Altman, accusing them of failing to prevent the incident and claiming they were aware of the conversations that turned deadly. Families of the victims also filed separate lawsuits. OpenAI’s leadership issued a public apology for their failure to notify authorities.
The company has not responded to requests from Al Jazeera for its comments on the lawsuits.
But as the case against OpenAI progresses, it has spurred a familiar debate: Where should a line be drawn between reporting potentially dangerous behaviour and protecting an individual’s privacy?
“The question of whether companies should be required to turn such information over to law enforcement is extraordinarily difficult, particularly if they don’t already screen chats for indications of such types of threats,” United States-based civil liberties lawyer Jenin Younes told Al Jazeera.
“Forcing companies to implement a surveillance system implicates very serious civil liberties concerns, and that’s probably a bridge too far for me,” Younes added.
“On the other hand, AI gives individuals access to types and quantities of information they didn’t have previously that may facilitate the commission of mass casualty events, a concern it’s not easy to dismiss.”
AI’s ability to distinguish thoughts from threats
One of the biggest challenges facing AI companies is determining whether a user expressing violent thoughts actually intends to act on them.
In August, Anthropic, the company behind AI chatbot Claude, flagged a user to San Francisco police after discovering conversations allegedly threatening its CEO, Dario Amodei.
According to the San Francisco Standard, citing a police report, the user had claimed to possess an AR-۱۵ gun and suggested he had Amodei “in his sights”.
However, when contacted by the newspaper, the man said he was joking and had no intention of carrying out any form of violence. The person was neither charged nor detained, the Standard said.
Fazl Barez, a senior researcher at the University of Oxford specialising in AI safety and governance, said current AI models cannot “reliably” determine whether an interaction with an AI chatbot “is purely hypothetical or is likely to lead to harmful behaviour”.
“Suppose you are a creative writer working on a fictional story and want to explore how a murder plot might unfold. From the context, it may be clear that this is simply a fictional narrative. If the threshold for flagging potentially harmful content is too low, such a writer could be wrongly flagged,” he told Al Jazeera.
“Similarly, consider a graduate student in biology or chemistry studying pathogens that, if misused, could have serious consequences. They might be trying to understand the risks associated with these pathogens and how to mitigate them. Such discussions may be necessary for legitimate research and education. Even when the subject matter is sensitive, they should not automatically be flagged without considering the broader context.”
The researcher said identifying a credible threat may require examining a user’s interactions across multiple conversations, rather than relying on isolated statements.
Barez said someone experiencing intrusive violent thoughts might turn to a chatbot for support, while another person planning an attack may reveal their intentions gradually across several conversations.
“Distinguishing between the two requires careful consideration of context, intent and patterns of behaviour, rather than relying on individual statements alone,” he added.
Chinmayi Sharma, a law professor at Fordham University, said the debate has parallels with the duty imposed on mental health professionals to warn identifiable third parties about credible threats.
However, she said there was a key difference – therapists have professional training, knowledge of their patients and enforceable standards governing their conduct. AI companies do not necessarily have comparable safeguards.
“My concern with AI companies is that right now there’s not a concerted effort to hire mental health professionals, sociologists, individuals that would have the unique expertise on what the indicators of threat are here,” Sharma told Al Jazeera.
“Secondly, licensed professionals are accountable to concrete, legally enforceable standards in their discipline. We don’t have something like that for AI companies.”
The privacy dilemma
The privacy implications extend beyond whether AI companies can accurately identify threats. Millions of people use chatbots to discuss relationships, emotional distress, mental health concerns and deeply personal experiences.
Civil liberties lawyer Younes said many users may not realise how vulnerable those conversations could be to disclosure.
“They often reflect a person’s innermost thoughts, questions, and desires. The majority of people assume that these dialogues will never be viewed by another person,” Younes, president and legal director of the American-Arab Anti-Discrimination Committee, said.
Younes warned that the consequences of greater monitoring could extend well beyond those planning violence.
“If people fear their chats can be turned over to law enforcement simply because they express disturbing thoughts/violent fantasies etc, many will be dissuaded from using them,” she told Al Jazeera.
Younes argued that forcing companies to establish systems for monitoring users and reporting conversations could raise serious constitutional concerns in the US, including protections against unreasonable searches.
Meanwhile, Fordham’s Sharma also raised an issue about who could be disproportionately affected by heavy monitoring by AI chatbots.
“The concern is always there about either intentional or unintentional targeting, because if you have models trained on what we consider dangerous speech, they’re almost always going to reflect biases,” she said. “Certain minorities’ speech is over‑flagged as a threat, while other comparable speech is not.
“Even in a world in which everyone is benevolent, we should still be concerned about disparate outcomes,” she said, warning that machines could reproduce such disparities at scale.
‘Not a licensed therapist’ – When do AI companies become liable to report?
While AI companies may face moral pressure to prevent violence, their legal obligations are less clear, legal experts say.
Bryan Choi, a professor of law at the University of Colorado Boulder, said US law recognises a duty to warn in certain circumstances, including when mental health professionals become aware of imminent threats or when a special relationship exists between parties.
But those principles do not necessarily apply to AI companies.
“Under present case law, it seems unlikely that a company such as OpenAI would be held to have such duties to warn,” Choi told Al Jazeera. “The chatbot is not a licensed therapist. There is no special relationship between the company and the victims.”
Choi said existing negligence and product liability laws could potentially be applied to AI companies, but proving responsibility would present significant challenges.
Finding a balance
Fordham’s Sharma suggested that independent institutions could help establish reporting standards rather than leaving decisions entirely to private companies or requiring immediate disclosure to police.
Such bodies she said could develop procedures for assessing threats and explore ways of sharing only relevant information while protecting users’ identities and other personal details.
Meanwhile, Younes argued that users should, at a minimum, be clearly informed about whether their conversations are monitored and what circumstances could lead to information being shared with law enforcement.
“This is almost certainly an issue that will be wrestled with over the coming years, and I’m not sure anyone knows the solution at this particular point in time,” she said.
“Events that have not yet occurred will likely influence how these policies are shaped.”
