this post was submitted on 20 Aug 2026
230 points (99.1% liked)

Technology

87390 readers
4115 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
all 46 comments
sorted by: hot top controversial new old
[–] pbjelly@sh.itjust.works 9 points 2 hours ago

TL;DR: AI transcript said I’d been a smoker since the age of 11, never smoked a day in my life.

My doctor asked me if she could use AI transcription software because she had hand surgery and couldn’t type. I said yes and reviewed the statements and while it was fairly accurate, it did have one egregious error on it.

When asked if I had ever smoked, in person, I replied with a simple “no,” and did not proceed to have any follow up conversation until her next question. The AI transcript for mysterious reasons decided my answer was “yes, since the age of 11.” 💀💀💀

Had to message them to fix it and while they did fix it asap, my first instinct wasn’t to check the transcript and I only did so out of curiosity. I can’t imagine what other people’s experiences may be.

[–] SCmSTR@lemmy.blahaj.zone 9 points 14 hours ago

Just say no.

We use a software to help us take notes is that okay? FUCK NO.

[–] sem@piefed.blahaj.zone 17 points 17 hours ago

Article:

Error by AI scribe during medical appointment leaves patient devastated

By the Specialist Reporting Team's Paige Cockburn

Posted 13 Aug 2026 at 2:55pm13 August 2026 at 2:55pm, updated 13 Aug 2026 at 4:04pm13 August 2026 at 4:04pm

A woman sits with her hands resting under her chin. She is wearing glasses and a colourful scarf and has short dark hair.

An error in Rebecca Green's medical records falsely claimed she had taken psychedelic mushrooms. (ABC News: Madeline Grace)

When Rebecca Green arrived for her first appointment with a urologist, she was asked whether she consented to having the appointment transcribed by artificial intelligence (AI).

She said yes without hesitation.

"I don't want to be in her bad books or be that annoying patient who takes up more of her time," she said.

But that decision would cause her a lot of stress, with the AI scribe making up the serious but false claim that Ms Green was taking illegal drugs.

"I was gobsmacked and in tears … it made up that I take [psychedelic] mushrooms. I have never done mushrooms in my life,"

she said.

More GPs and hospitals are using AI scribes. What are the risks?

doctor generic

AI tools are becoming more prevalent in healthcare, helping overworked doctors to carry the workload. But it has come with concerns.

New research by a privacy watchdog has raised serious concerns about AI being used in medical settings due to its propensity to "hallucinate" false symptoms or conditions, and warned patients were at risk.

Ms Green only picked up the mistake after her kidney stone surgery in March, when she read the post-operative letter her specialist sent to her GP.

It stated Ms Green micro-dosed mushrooms and that could be the reason for prior bleeding around the kidneys.

"For this letter to say something that wasn't based in any evidence but could potentially have huge consequences for me … it was quite scary," she said.

"It felt very impersonal to me. I wasn't getting the level of care and familiarity from the doctor that I would be hoping for in such a vulnerable situation."

A woman wearing glasses and a black sleeveless top sits looking at a piece of paper with writing on it. Kitchen behind her.

Rebecca Green discovered the error on her medical record after reading a post-operative letter from her specialist.  (ABC News: Madeline Grace)

Ms Green was particularly stressed as she was receiving workers' compensation and felt any mention of illegal drugs on her medical record could impact her case.

Australia's peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations. Ms Green said it was clear her doctor did not do this.

"If you had checked that letter you would have been able to correct that before it was sent out … why aren't you in tune with each of your patients and what's going on?"

What to know before saying yes to your doctor using AI

A generic image of a female doctor talking to an unidentifiable male patient

Health practitioners are using generative AI to help them record and summarise patient information, but some patients are saying no.

After Ms Green made a complaint, the urologist sent a letter apologising.

"Please be assured that we take the accuracy of our documentation very seriously, and we are truly sorry that this occurred," the letter stated.

The doctor said she had not been able to determine how the claim about mushrooms came to be included, but it appeared to be from an error during the "dictation or transcription process".

The urologist corrected the correspondence and in her letter said she would review how AI was used in her practice.

The doctor did not provide any responses to the ABC's questions about what changes, if any, have been made since.

AI confuses which breast has cancer

Ms Green feels lucky she spotted the error, but is worried some people might not be aware of mistakes on their medical record, which could have repercussions for their health, their career or their insurance.

This week the ABC revealed instances where private health insurers were asking providers for their patients' full clinical history as part of "aggressive" auditing practices.

AI scribes have become prevalent within the healthcare sector. The Royal Australian College of GPs (RACGP) last year estimated that 40 per cent of GPs were regularly using them, but that number was considered conservative.

A woman with her tied back is seen from behind, she is facing a man in a suit who is sitting in front of a computer.

The peak body for GPs in Australia has estimated that 40 per cent are regularly using AI scribes at work. (ABC News: Tim Swanston)

Research shows having AI transcribe an appointment can reduce the administrative burden for GPs and allow them to focus more on their patient, rather than the computer.

But for every benefit, there are risks.

A recent report by advocacy group Digital Rights Watch (DRW) found Australians were being exposed to unregulated AI medical scribes, sometimes without their consent, and errors had the potential to change clinical decision-making.

Do you have a story to share about AI in health care?

Email Paige Cockburn at cockburn.paige@abc.net.au

Examples of mistakes found in the DRW report included a case where a scribe recorded the wrong breast in a diagnosis of breast cancer and said someone had epilepsy, when they did not.

Chief executive of the Consumers Health Forum of Australia Elizabeth Deveny said she often heard complaints of AI mixing up words and patients only finding out by chance.

"People report both frustration in finding the original source of the error and then getting the record changed," she said.

Profile photo of a woman with short blond hair in a green blazer and white shirt and red lipstick.

Elizabeth Deveny says it is frustrating for patients who have to get their records fixed when errors are made. (ABC News: Darryl Torpy)

Dr Deveny, who is currently involved in a major research project on AI medical scribes, said doctors may be too trusting of AI.

"There's evidence that for the first few weeks of using AI, [people] check things pretty carefully and then at a point they stop checking because they assume it will be right."

Perth GP Sean Stevens, who has been using a scribe for three years, said it was incumbent on the doctor to check every transcript, but believed the technology was becoming more accurate.

"Three years ago, I would have to correct multiple things on each transcription. These days it's like one correction every third or fourth transcription."

A man wearing a dark coloured shirt smiles with his mouth open. He has short dark, cropped hair and there are plants behind him.

Sean Stevens says AI transcriptions are getting more accurate, but doctors still need to check them. (Supplied)

Dr Stevens is a big advocate of tech; he is the RACGP's Digital Health and Innovation chair as well as an advisor for one of the biggest AI scribes, but he admits AI has its weaknesses and can produce incorrect results that sound plausible.

"The classic is, it will get the side of the body incorrect and say right when you've said left … so you need to check things pretty closely, especially things like drug doses,"

he said.

He said using AI did not mean he could squeeze more patients in, but it allowed him to run on time more often and produce more thorough referrals.

Consenting to AI's involvement

The DRW found huge discrepancies in how consent is being obtained, with some doctors only putting a sign in the waiting room, which does not meet privacy law requirements.

Other times consent must be given when the appointment is booked online, and alarmingly some patients are even being refused appointments if they do not consent.

Dr Deveny said consent must be enthusiastic and fully informed, and patients should feel comfortable asking questions, even if it feels intimidating.

What the Consumer Health Forum recommends asking at your next medical appointment:

  • Are you using AI today and if so, what are you using it for?
  • Will the doctor review everything before it goes into my record?
  • Will I be able to get a copy of the notes if I want them?
  • What's the process if I think something's wrong?

The DRW report also identified that people might not be forthcoming about sensitive issues, like family violence, when they know AI is listening.

Dr Stevens said that was why he asked for consent in every appointment, even with regular patients.

"It's very important patients disclose any and all things that are relevant to their health … and that they know they can say no [to AI] … as a doctor you have to respect that," he said.

TGA flags breaches will be dealt with

There are around a dozen AI medical scribes available in Australia, but none are approved by the regulator — the Therapeutic Goods Administration (TGA) — because if the product claims to be solely for transcription and summarising, it is exempt from oversight.

The TGA only considers tools that do things like suggesting a diagnosis or treatment plan as a "medical device".

AI changing the way GPs tend to their patients

A doctor talks to a patient in an office.

Some GPs are now relying on artificial intelligence to take notes during patient consultations. So, how safe is our sensitive information?

Currently, no AI scribes have gone through the regulation process, despite some claiming to offer clinical suggestions.

The TGA has acknowledged this means some scribes were potentially in breach of the law.

The TGA has spent the last year reviewing AI scribes and says its findings will be released in the coming weeks, and any breaches will be acted upon.

Chief executive of Patients Australia Lisa Robins believed all scribes should be regulated, as summarising a patient still involved decision-making by AI.

Dr Stevens had concerns that extending regulation would only stifle future innovation.

The DRW is calling on the newly established AI Safety Institute to test AI scribes and publish a list of approved tools.

This would be similar to New Zealand's approach, where the national AI advisory group reviewed tools and last year gave approval for clinicians to use two AI tools: Heidi Health and iMed.

Ms Deveny said ensuring doctors were using AI safely could also be enforced in other ways, such as compulsory checks and balances built into the software, or during practice accreditation, which happens every three years.

Portrait of a woman with short brown hair she is wearing glasses and a colourful scarf and looks straight ahead, non-smiling.

The error on her medical record has left Rebecca Green reconsidering if she would give consent to AI transcription again.  (ABC News: Madeline Grace)

Ms Green said her trust in AI has been splintered by what happened and now would only give consent for AI if she had an established relationship with the doctor.

"I love AI … but the essential ingredient is the human. At the end of the day what's happened here is more about the lack of human involvement,"

she said.

[–] Abyssian@lemmy.world 13 points 20 hours ago

The patient was devastated? You should have seen how many times the AI apologized.

[–] Killer57@lemmy.ca 27 points 1 day ago (2 children)

As far as I'm concerned, using AI of any sort inside of a doctor's office is a grave violation of HIPAA

[–] MangoCats@feddit.it 1 points 19 hours ago (1 children)

Let me tell you about how medical transcriptionists used to do it: Sister in law was one, she didn't seem to care about HIPAA too much, she and all her colleagues did lose their jobs about a 15-20 years ago when AI transcription software took over their jobs, but before that she was frequently given jobs because her predecessors made serious mistakes in their transcriptions.

[–] sem@piefed.blahaj.zone 7 points 17 hours ago (1 children)

If she didn't care about HIPAA, I'm glad she lost the job lol.

If only we could fire AI too.

[–] MangoCats@feddit.it 1 points 2 hours ago (1 children)

She'd tell stories (without the names)- many of her colleagues would get deep into the personal details...

[–] sem@piefed.blahaj.zone 1 points 2 hours ago (1 children)

I don't think that's a hipaa violation, but I'm no expert.

EMTs, etc, do that too. You'd go crazy if you couldn't.

[–] MangoCats@feddit.it 2 points 1 hour ago

I work in the digital handling of PHI (HIPAA regulated) it's pretty insane what is considered HIPAA protected in that world.

In practice, yeah, real people can't just be all "oh, that's a national secret, I can't tell you anything about my work or the weird worms people had in weirder places...

[–] LordCrom@lemmy.world -4 points 18 hours ago (2 children)

Ai transcription is everywhere. You cant get away from it

[–] SCmSTR@lemmy.blahaj.zone 5 points 14 hours ago* (last edited 14 hours ago)

Yeah? Watch me.

Watch me and millions like me push back everywhere and slowly change the world.

[–] Akuchimoya@startrek.website 10 points 17 hours ago (1 children)

Speech-to-text (a type of AI) has been around for a fair good years, but now they're being paired with generative AI. This is the kind of thing that shouldn't be paired with a generative LLM. A language model to identify words? Yes. Generative? No.

[–] ChaoticNeutralCzech@feddit.org 3 points 14 hours ago

Yes. It technically does improve accuracy (homophone correction, for example) but the occasional errors are worse and way less obvious.

[–] phoenixz@lemmy.ca 47 points 1 day ago (3 children)

Australia's peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations.

Show me one single AI user that checks the output of their AI, and I will show you my unicorn

[–] luciferofastora@feddit.org 4 points 14 hours ago

I just asked SlopGPT a question and checked the output. Can I see your unicorn?

(That was a lie. I just really want to see a unicorn.)

[–] quantumcrop@lemmy.today 10 points 22 hours ago

Not gunna get me with the ole "wanna see a unicorn" trick twice.

[–] Initech_vs_Initrode@lemmy.zip 2 points 15 hours ago

Luckily, I know something that would probably help her get over the trauma...

[–] kambusha@sh.itjust.works 8 points 1 day ago (1 children)

I read "medieval" appointment and was thoroughly confused, and intrigued.

[–] postman@literature.cafe 14 points 1 day ago* (last edited 1 day ago) (3 children)

I've used AI for medical transcription before and it is absolutely incredible. It's not just a word for word text, but a detailed clinical breakdown of everything discussed.

All you have to do is simply read the damn thing thoroughly and correct whatever minor errors pop up.

Takes 95% less time.

[–] aeronmelon@lemmy.world 23 points 1 day ago

All you have to do is fire the human who is trained and rarely makes a mistake and replace them with a paid machine that almost always makes a mistake and double check everything yourself (or hire another human to double check it) because someone’s life literally depends on the entirety of the text being absolutely correct.

AI does work at a level that makes unpaid interns look like career professionals. All for the low cost of completely fucking the environment.

Glad you like it.

[–] Akasazh@lemmy.world 47 points 1 day ago (3 children)

I'm unsure if there's an ethical issue with putting patients medical records in the data machine.

[–] panda_abyss@lemmy.ca 20 points 1 day ago

There exists online and offline ML.

Transcription is not a task that needs a full blown LLM, summarization probably does, and if being done offline or on a server with trusted verifiable lack of logging should be fine. That does not mean send it to ChatGPT API.

I would hope they store the raw conversation file encrypted with patient records so they can retranscribe later as tools improve.

[–] EvilBit@lemmy.world 5 points 1 day ago* (last edited 1 day ago) (2 children)

Medical AI has guardrails out the wazoo. By the time it’s made available to a physician, it’s probably been reviewed by a council, vetted by IT, and negotiated heavily by Legal. If the data leaves the premises, it’ll be thoroughly protected and isolated.

That said, fuck AI. I’d give up medical transcription to get the environment, economy, and sanity back. Musk’s data center is running dozens of natural gas turbines in a low-income, primarily Black neighborhood 11 miles down the road from St. Jude Children’s Research Hospital. It’s positively ghoulish.

[–] partofthevoice@lemmy.zip 6 points 18 hours ago* (last edited 17 hours ago) (1 children)

Horseshit. Medical AI is using Oracle services. Larry mother fucking Elison is storing and processing this data.

The medical industry seems so ignorant about this. They think the argument is just about AI. It’s also about the fact that I don’t trust the people you’re partnered with.

I get so pissed when the AI consent is baked into other paperwork with no opt out. I will leave.

Edit: https://www.oracle.com/customers/beacon-health/

Edit: they might not all be, but I’m not authorizing your arbitrary “AI use” clause. You might as well be, now or at some point down the road.

[–] EvilBit@lemmy.world 1 points 15 hours ago

The agreement is only as good as the hospital’s vetting, so I may be a little optimistic after seeing a very thorough and protracted process on my side. If you work for a healthcare system that’s okay with Beacon and Oracle, then it may be time to evaluate whether they’re a good healthcare system to work for.

[–] MangoCats@feddit.it 6 points 19 hours ago* (last edited 19 hours ago)

Clue: Medical transcription has been done by "AI" for the past 20 years. Source: sister in law was a medical transcriptionist making $80K per year, until Dragon Naturally Speaking and friends decimated her job market.

[–] GamingChairModel@lemmy.world 4 points 1 day ago

It needs to be a system and a contractual framework that complies with the medical privacy regulations of the place where it's being used (in the US, HIPAA). I have some doubts about whether specific companies or people will actually comply, or the enforcement of those laws by certain governmental authorities, but I don't think it's conceptually impossible to have this kind of tech be compliant with those types of regulations.

[–] halcyoncmdr@piefed.social 21 points 1 day ago (2 children)

Exactly. It is a tool for professionals to use to speed up existing processes. It is not a replacement for people, regardless of what the marketers tell you so you buy their product. I don't understand how there aren't thousands of lawsuits against these AI companies for their blatantly false advertising.

AI keeps proving this day by day when people try to make it a replacement and have it blow up in their faces every day.

[–] greyscale@lemmy.grey.ooo 3 points 1 day ago (1 children)

blatantly false advertising.

SEC is neutered, what makes you think anyone in charge of advertising rules hasn't been replaced by a 17 year old called Goebbels Gobbler or something with a DOGE laptop.

[–] halcyoncmdr@piefed.social 7 points 1 day ago (1 children)

SEC is neutered

This article is about an incident in Australia. The SEC has no bearing on what Australia can do.

[–] greyscale@lemmy.grey.ooo 2 points 19 hours ago

The AI bubble is an American phenomenon and their SEC isn't slowing down the insanity. Their lack of a functional SEC is everyones problem.

[–] First_Thunder@lemmy.zip -5 points 1 day ago (2 children)

Hell, I just found out a few months ago my father’s office (he’s a lawyer) still did transcriptions of court proceedings manually! Full blown lawyers wasting expensive man hours doing that. As soon as I found out I got some random whisper cpp frontend and slapped it on a random computer

[–] halcyoncmdr@piefed.social 7 points 1 day ago (1 children)

As long as they still have someone verifying and correcting mistakes, that's a decent solution. It doesn't need to be complicated. Whisper is by no means perfect, but from my experience it handles 90% fine, only having to listen and correct 10% is a huge improvement over transcribing everything manually.

The issues with so much of the current AI use is implementations that completely remove trained human oversight from the equation because the AI companies position it as a complete replacement for employees.

[–] prole@lemmy.blahaj.zone 2 points 1 day ago (1 children)

from my experience it handles 90% fine, only having to listen and correct 10% is a huge improvement over transcribing everything manually.

I'm curious how this is supposed to work? How do you only listen to 10%? How are you supposed to know which 10% to listen to? Wouldn't you still need to listen to it all to find that 10%?

[–] emmanuel_car@fedia.io 4 points 1 day ago

Yes, but listening to it all once and correcting the mistakes as you hear/see them would be faster than transcribing the entire thing, then proof reading.

[–] prole@lemmy.blahaj.zone 3 points 1 day ago

I hope his clients are aware