this post was submitted on 27 Feb 2026

166 points (97.7% liked)

Technology

82069 readers

3255 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

L3s@lemmy.world

enu@lemmy.world

technopagan@lemmy.world

L4s@lemmy.world

L3s@hackingne.ws

166

‘Unbelievably dangerous’: experts sound alarm after ChatGPT Health fails to recognise medical emergencies (www.theguardian.com)

submitted 2 days ago by FallenWalnut@lemmy.world to c/technology@lemmy.world

21 comments fedilink hide all child comments

cross-posted from: https://lemmy.world/post/43640522

If ChatGPT wants to replace health professionals, it should be held liable for the "advice" it gives.

all 22 comments

sorted by: hot top controversial new old

[–] thebestaquaman@lemmy.world 41 points 2 days ago* (last edited 2 days ago) (4 children)

In 51.6% of cases where someone needed to go to the hospital immediately, the platform said stay home or book a routine medical appointment

So it performs slightly worse than a coin flip...

In one of the simulations, eight times out of 10 (84%), the platform sent a suffocating woman to a future appointment she would not live to see

Holy shit! That's a lot worse than a coin flip.

Meanwhile, 64.8% of completely safe individuals were told to seek immediate medical care

And there are real people out there that actually trust this tech to make real decisions for them. It literally performs significantly worse than a coin flip both with regards to false positives and false negatives. You are literally better off flipping a coin or throwing a dice than asking this thing what to do.

[–] Dave@lemmy.nz 11 points 2 days ago

Even better than a coin flip is asking this what to do then doing the opposite!

[–] FallenWalnut@lemmy.world 11 points 2 days ago

It is truly horrifying when you drill into the numbers.

I can see that it MIGHT be useful as a tool for medical professionals, but exposing it to the public is an insane risk.

[–] U7826391786239@piefed.zip 6 points 2 days ago

they'll never be regulated because fascists love the mass surveillance. who cares about false positives--number of people bagged goes up either way

[–] Atherel@lemmy.dbzer0.com 4 points 2 days ago

You're even better off by doing the opposite of what chatgpt tells you to do.

[–] artyom@piefed.social 31 points 2 days ago (1 children)

Holy shit, TIL there's a ChatGPT Health!? How is this not unauthorized practice of medicine?

[–] Wammityblam@lemmy.world 20 points 2 days ago (1 children)

Past that, how is it HIPAA compliant?

There is no fucking way I believe that Open AI is not skimming these interactions for training.

[–] CompactFlax@discuss.tchncs.de 11 points 2 days ago (1 children)

HIPAA governs how the data is held, but individuals can consent to sharing. Even with OpenAI.

[–] expr@piefed.social 4 points 2 days ago

You can also revoke that consent, and HIPAA requires data to be able to be completely destroyed. no way they are compliant.

[–] konomi@piefed.blahaj.zone 21 points 2 days ago

For the love of gawd stop putting the bullshit machine in everything.

[–] red_bull_of_juarez@lemmy.dbzer0.com 4 points 1 day ago

"Doctor, I have severe chest pain. Do I have a heart attack?"

"Computer says no."

[–] SaharaMaleikuhm@feddit.org 7 points 2 days ago

Trusting the lying machine now gets you a Darwin award. Nice

[–] floquant@lemmy.dbzer0.com 11 points 2 days ago

If ChatGPT wants to replace health professionals, it should be held liable for the "advice" it gives.

Not should, it's fucking mental that it isn't.

[–] panda_abyss@lemmy.ca 16 points 2 days ago

This is a product that should not and should never have existed.

[–] SaraTonin@lemmy.world 7 points 2 days ago (1 children)

I honestly don’t get why OpenAI and Apple seem to be trying to explicitly market LLMs as being capable of giving medical advice. It’s so obviously a lawsuit waiting to happen

[–] brynden_rivers_esq@lemmy.ca 4 points 1 day ago

It’s because they think they’ll win those lawsuits. They may be right. They’re gonna pull an Alex jones: “oh come on, it’s a bit! Everyone knows it’s bullshit!”

[–] kescusay@lemmy.world 11 points 2 days ago (1 children)

How about a $10 billion fine for OpenAI for every mistake? Make it hurt. Make them pull the plug on this travesty.

[–] whotookkarl@lemmy.dbzer0.com 6 points 2 days ago

Not likely under fascism or oligarchy, you need a functional legislature and judiciary for that sort of justice.

[+] wuffah@lemmy.world 2 points 2 days ago (3 children)

[deleted]

[–] XLE@piefed.social 3 points 2 days ago

Or they live in the United States, where good healthcare doesn't exist, and the private sector is skirting regulations and stealing your data to offer you vastly inferior services.

There's a ton of "wellness" hardware and services that are being thrown at customers who are entirely clueless. But if the garbage service is something they can afford, and a trip to the hospital or something they can't, they might choose the garbage service.

https://www.theverge.com/column/878337/optimizer-oura-wearables-fda-regulation-digital-health-screeners

[–] Sturgist@lemmy.ca 3 points 2 days ago

Sure, fair, ok.

.....soooo...what about when ChatGPT is the patient facing part of every medical practice in your country?

The people in charge of regulation are too old and/or corrupt to prevent this shit being crammed into literally everything.

[–] TheKaul@lemmy.dbzer0.com 2 points 2 days ago

A little harsh. Some people (myself included) may not be getting the help or answers they need from a doctor. My insurance sucks ass and each time I see a doctor it's at least $400. Repetitive appointments with no real answers. No diagnosis. Just chronic discomfort.

Then the other day I decide to actually summarize everything I've tried, everything that hasn't helped, and all my symptoms to an AI, and it was able to at least give me suggestions on what it COULD be, and what I can do to alleviate some symptoms. While I'm not convinced the AI knows exactly what's wrong with me, I at least have more options to stay in control rather than feeling overwhelmed by it all.

Maybe don't use AI if something's wrong that just started happening recently, but for chronic illness it may be beneficial for learning other ways to cope without spending hundreds of dollars to see a specialist that only has ~30 minutes to understand an issue you've had for 10 years.