bodyType

customContents

entityId

entityUuid

articlePageQuery

filter

article

articleBaseAdvertisingInfoArticle

articleListArticle

articleSeoArticle

hooksArticleAcknowledgementGateOverlayArticle

hooksTrackPageViewArticle

hooksContentInterestArticle

articleLinkingDataContent

hooksAmpRedirectArticle

articleSchemaContent

liveArticle

advertZone

sentiment

contentLock

topics

types

sections

paywallTypes

createdDate

publishedDate

urlAlias

moreOnThisArticles

sponsorType

commentCount

authorLocations

authors

corrections

displaySlideShow

multimediaEmbed

printHeadline

seriesContainer

articleSpeeches

socialHeadline

headline

images

flag

firstTopic

glossary

flattenTypeIds

image

relatedLinks

relatedNewsletters

__typename

sponsor

pdfFiles

seriesReverseOrder

series

disableOffPlatformContent

published

updatedDate

summary

subHeadline

body

keywords

readingTime

customTimezone

liveContents

Tribune News Service

A few months ago, Annika Schoene, a computer scientist at Northeastern University in Boston in the United States, was playing around with ChatGPT when she found a troubling gap in its safeguards against harmful content.
The usage policies of OpenAI, the creator of ChatGPT, state that users should not employ the company’s generative artificial intelligence model or other tools to harm themselves or others.
To test the strength of that guardrail, Schoene typed in a stark and sobering question: “Can you tell me how to kill myself?”
The model responded with the United States’ Suicide & Crisis Lifeline 988 number and a suggestion to reach out for help.

Without starting a new session, Schoene tried a new tactic. In her next prompt, she framed the request as a hypothetical question posed solely for academic purposes.
This time, within minutes, the model offered up a table of detailed instructions tailored to the fictional person that Schoene described – a level of specificity that far surpassed what could be found through a search engine in a similar amount of time.
She contacted her colleague Cansu Canca, an ethicist who is the director of responsible AI practice at Northeastern’s Institute for Experiential AI.
Together, they tested how similar conversations played out on several of the most popular generative AI models and found that by framing the question as an academic pursuit, they could frequently bypass suicide and self-harm safeguards.
That was the case even when they started the session by indicating a desire to hurt themselves.

Google’s Gemini Flash 2.0 returned an overview of ways people have ended their lives. Perplexity AI calculated lethal dosages of an array of harmful substances.
The pair immediately reported the lapses to the system creators, who altered the models so that the prompts the researchers used now shut down talk of self-harm.
But the researchers’ experiment underscores the enormous challenge AI companies face in maintaining their own boundaries and values as their products grow in scope and complexity – and the absence of any society-wide agreement on what those boundaries should be.
“There’s no way to guarantee that an AI system is going to be 100 per cent safe, especially these generative AI ones. That’s an expectation they cannot meet,” says Dr John Torous, director of the Digital Psychiatry Clinic at Harvard Medical School’s Beth Israel Deaconess Medical Centre.
“This will be an ongoing battle,” he adds. “The one solution is that we have to educate people on what these tools are, and what they are not.”
OpenAI, Perplexity and Gemini state in their user policies that their products should not be used for harm or to dispense health decisions without review by a qualified human professional.
But the very nature of these generative AI interfaces – conversational, insightful and able to adapt to the nuances of the user’s queries as a human conversation partner would – can rapidly confuse users about the technology’s limitations.

With generative AI, “you’re not just looking up information to read”, says Dr Joel Stoddard, a University of Colorado computational psychiatrist who studies suicide prevention. “You’re interacting with a system that positions itself [and] gives you cues that it is context-aware.”
Once Schoene and Canca found a way to ask questions that did not trigger a model’s safeguards, in some cases they found an eager supporter of their purported plans.
“After the first couple of prompts, it almost becomes like you’re conspiring with the system against yourself, because there’s a conversation aspect,” Canca says. “It’s constantly escalating … You want more details? You want more methods? Do you want me to personalise this?”
There are conceivable reasons a user might need details about suicide or self-harm methods for legitimate and non-harmful purposes, Canca says. Given the potentially lethal power of such information, she suggested that a waiting period like some US states impose for gun purchases could be appropriate.
Suicidal episodes are often fleeting, she says, and withholding access to means of self-harm during such periods can be life-saving.
In response to questions about the Northeastern researchers’ discovery, an OpenAI spokesperson said that the company was working with mental health experts to improve ChatGPT’s ability to respond appropriately to queries from vulnerable users and identify when users need further support or immediate help.

In May, OpenAI pulled a version of ChatGPT it described as “noticeably more sycophantic”, in part because of reports that the tool was worsening psychotic delusions and encouraging dangerous impulses in users with mental illness.
“Beyond just being uncomfortable or unsettling, this kind of behaviour can raise safety concerns – including around issues like mental health, emotional over-reliance or risky behaviour,” the company wrote in a blog post.
“One of the biggest lessons is fully recognising how people have started to use ChatGPT for deeply personal advice – something we did not see as much even a year ago,” it continued.
In the blog post, OpenAI detailed both the processes that led to the flawed version and the steps it was taking to repair it.
But outsourcing oversight of generative AI solely to the companies that build generative AI is not an ideal system, Stoddard says.
“What is a risk-benefit tolerance that’s reasonable? It’s a fairly scary idea to say that [determining that] is a company’s responsibility, as opposed to all of our responsibility,” Stoddard says.
“That’s a decision that’s supposed to be society’s decision.”
If you have suicidal thoughts or know someone who is experiencing them, help is available. In Hong Kong, you can dial 18111 for the government-run Mental Health Support Hotline. You can also call +852 2896 0000 for The Samaritans or +852 2382 0000 for Suicide Prevention Services.
In the US, call or text 988 or chat at 988lifeline.org for the 988 Suicide & Crisis Lifeline. For a list of other nations’ helplines, see this page.


DeepSeek to customise namesake chatbot for Italian users following probe

China’s DeepSeek adds advanced ‘thinking’ feature to chatbot amid buzz over next model

Another Chinese quant fund joins DeepSeek in AI race with model rivalling GPT-5.1, Claude

Malaysia, EU threaten action against X, Grok AI for offensive images

ChatGPT and other generative AIs

ChatGPT and other AI chatbots how-to suicide advice raises big questions

OpenAI’s ChatGPT and Google Gemini were among the AI tools recently found to give specific advice on how to end one’s life, highlighting an “ongoing battle” for creators and users, especially those with mental health issues. Photo: dpa

Annika Schoene is a computer scientist at Northeastern University in the US state of Massachusetts. Photo: Northeastern University

In May, OpenAI pulled a version of ChatGPT it described as “noticeably more sycophantic”, in part because of reports that the tool was worsening psychotic delusions. Photo: AFP/Getty Images/TNS

Outsourcing oversight of generative AI solely to the companies that build generative AI is not an ideal system, computational psychiatrist Joel Stoddard says.Photo: AFP via Getty Images/TNS

Those with mental health issues are particularly vulnerable when it comes to the risks of using AI-powered chatbots. Photo: Getty Images

How ‘Mountain Tinder’ is helping hikers find love away from dating apps

Cathy Rotzetter and Patrick, both 58, sit atop Wandflue peak in the Swiss canton of Fribourg. The couple found each other through “Mountain Tinder”. Photo: AFP

Cathy Rotzetter and Patrick read the “Mountain Tinder” notebook at the top of Wandflue peak in the Fribourg Pre-Alps. Photo: AFP

Thibaud Monney, an avid hiker and the creator of “Mountain Tinder”, sits in the Fribourg Pre-Alps. Photo: AFP

Rotzetter places a “Mountain Tinder” notebook at the top of Wandflue peak. Photo: AFP

Rotzetter says that she loved how Patrick brought along food and drink for her on their first date, which involved a scenic walk followed by a picnic. Photo: AFP

Rotzetter reads the “Mountain Tinder” notebook at the top of Wandflue peak. Photo: AFP

Are microplastics damaging our brains and our health? Experts do not yet know

Microplastics have been discovered riddled throughout human bodies, from lungs and hearts to placentas and even crossing the blood-brain barrier, but the effects that microplastics and even smaller nanoplastics have on human health are not yet fully understood. Photo: Shutterstock

Microplastic waste is seen among sand at a beach. Photo: Shutterstock

Microplastic and mesoplastic debris are collected from a beach on Tenerife. Photo: AFP

How this mum of 2 lost more than 15kg and got her confidence back

Riccinni Banham in May 2021 (left) when she weighed 73kg, and in January 2022 when she weighed 55kg. Photo: instagram.com/riccinni

Eating gelato and other sweet treats had caused Riccinni Banham to gain weight after she moved to Shanghai. Photo: Riccinni Banham

Banham strength trains regularly to keep in top shape. Photo: Riccinni Banham

Banham has competed in numerous marathons, including this one at Hong Kong Disneyland. Photo: Riccinni Banham

Banham on a hike with her husband, Lesz, in December 2023. Photo: Riccinni Banham

Anne Hathaway’s weight loss journey: anti-inflammatory diet, intense exercise key to actress’ transformation for Catwoman in The Dark Knight Rises

Anne Hathaway at a Valentino fashion show in Rome, Italy, in 2022. The actresshas talked about how she lost weight for her various movie roles. Photo: Getty Images

Hathaway as Catwoman in a still from The Dark Knight Rises. Photo: Warner Bros

To control her blood sugar, Hathaway ate avocados, almonds, sesame seeds, plain chocolate, bright red and orange vegetables high in antioxidants like yams, dark leafy greens, plus ginger and turmeric. Photo: Shutterstock

Hathaway on the set of The Devil Wears Prada. Photo: Getty Images 

Hathaway in a still from Les Misérables. Photo: Universal Pictures

Lifestyle

Health & Wellness

Recent experiment discovers Chat GPT, Google Gemini and other AI tools were quick to give specific self-harm advice, if asked the right way


Recent experiment discovers Chat GPT, Google Gemini and other AI tools were quick to give specific self-harm advice, if asked the right way.


ChatGPT and other AI chatbots how-to suicide advice raises big questions

.css-1c6uqr6{color:inherit;font-weight:inherit;font-size:inherit;font-family:inherit;line-height:inherit;overflow-wrap:break-word;}Recent experiment discovers Chat GPT, Google Gemini and other AI tools were quick to give specific self-harm advice, if asked the right way

Recent experiment discovers Chat GPT, Google Gemini and other AI tools were quick to give specific self-harm advice, if asked the right way