this post was submitted on 15 Nov 2026
607 points (98.3% liked)

Technology

89869 readers
3550 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] beejjorgensen@lemmy.sdf.org 421 points 2 days ago (5 children)

No mails arrived due to the disruption [...]. So no one noticed that the maintenance safety system of the sauna landscape has long since sounded the alarm by mail. Because the automatic dosing system had pumped too much chlorine into the pool.

Kinda scary to rely solely on email for such alarms.

[–] Remember_the_tooth@lemmy.world 229 points 2 days ago (7 children)

Yeah, I hate Microsoft as much as the next person, but this is 100% on Phantasialand for trying to cut corners with excessive automation.

[–] 2ndDegreeOWO@lemmy.world 99 points 2 days ago (2 children)

It's not excessive automation -- it's a lack of redundancy for critical safety features.

Anything safety-critical should never have a single point of failure if you can help it. And there was definitely an opportunity here to have some kind of secondary backup alarm system.

[–] raspberriesareyummy@lemmy.world 34 points 2 days ago

Nothing safety critical should ever go through microslop

[–] Schemen110676@lemmy.ml 10 points 1 day ago

On top of that it should fail safely.. and not simply stop working without anyone knowing

[–] tal@lemmy.today 117 points 2 days ago* (last edited 2 days ago) (1 children)

I think that automation is fine, but when you start talking about life-critical systems, the bar on failure modes and reliability becomes a lot higher. Like, if you have a sensor for excessive chlorine levels, the default probably should be not to keep pumping more of the chemical in, but to have some "hard" upper level where no more chlorine goes in (and if the two systems aren't directly-integrated, to at least let the maintenance safety system power off the chlorine system), and for a secondary alarm to be triggered (like, start playing a loud audible alarm) if maintenance doesn't respond to the email notification and indicate that they've dismissed the alert within a given amount of time. That's disruptive and undesirable compared to just emailing maintenance, but as long as there aren't any failures, you shouldn't ever see it. But if you do have it come up, you don't want the failure mode to be chemical-keep-being-dumped-into-the-system-and-people-being-unaware-of-it.

You want fail-safe behavior.

Automation can increase safety by decreasing time until an issue is addressed, for example. Automation isn't intrinsically incompatible with safety. Aircraft computers today use TCAS to automatically instruct pilots what to do if they are about to collide (and in fact, today the automated system is explicitly given higher priority than human air traffic controllers


if the two give pilots conflicting directives, the human is the one to be disregarded). But you gotta make your automated procedures have high reliability and handle failure the way you want when the consequences of a failure are high (and that goes for non-automated procedures too).

[–] panda_abyss@lemmy.ca 7 points 1 day ago

It’s like those supermarket fridges.

When these things go off a literal alarm needs to be screaming that there’s an issue.

Yes, you can set up additional email alerts and logging. But the primary mechanism needs to be an in your face loud uncomfortable alarm, and safety cutoffs like you said.

[–] Schemen110676@lemmy.ml 15 points 1 day ago (1 children)

No.. with the wrong kind of automation.

Safety relevant systems do not belong on public infrastructure

[–] NonconfrontationaI@lemmy.ml 4 points 1 day ago

Outlook isn't public infrastructure lmao.

[–] assaultpotato@sh.itjust.works 59 points 2 days ago (1 children)

Excessive automation? That actually seems like a great thing to have automated... just not by email. Nothing stopping them from having a sub ghz radio that sets off a physical alarm somewhere. Improper automation I think would be more apt. A purely local automated alarm sounds great, but the internet shouldn't be involved imo, at least not on the critical path to remediation.

[–] db2@lemmy.world 10 points 1 day ago (1 children)

just not by email

I disagree, email definitely had a place. Documenting the fault and time it happened.

The way they did it was stupid.

[–] assaultpotato@sh.itjust.works 10 points 1 day ago

Sure, for record keeping or for distribution.

But not for safety, durable delivery or for actually taking action off of.

[–] helpImTrappedOnline@lemmy.world 25 points 2 days ago* (last edited 2 days ago)

I wonder if they could have done something like [puts on big brain hat] have an actaul fucking alarm the pool staff and guests can hear.

[–] Miaou@jlai.lu 5 points 1 day ago (1 children)

"Excessive"?! You mean lack thereof? Do you understand emails are not automation?!

Poorly designed automation maybe.

Hope these fuckers get absolutely destroyed in the neglect suit.

[–] SeeMarkFly@lemmy.ml 9 points 2 days ago

Well, at least the automation was not artificially intelligent.

[–] lornosaj@lemmy.world 18 points 1 day ago

In proper SRE, there is a rule - mails are not alerts and must not be used as such.

[–] Fmstrat@lemmy.world 4 points 1 day ago

100%. This is not because of an outage, this is because whoever set this up is an idiot.

[–] Reannlegge@lemmy.ca 15 points 2 days ago (1 children)

I for one think that IT team should be kicked to the curb, but they will probably get raises to figure out that they can use email, healthchecks.io, and ntfy all at the same time for redundancy or they will say we need money to switch providers and set up a gmail account to do the same thing and call that redundant cause you know getting two emails saying the same thing is not going to end up just being ignored.

[–] shalafi@lemmy.world 1 points 2 days ago (3 children)

I used to have IT monitors that would email but also text me. Most folks don't know they can send an email to (phone_number)@(provider).com and that shows up as a text.

[–] halcyoncmdr@piefed.social 8 points 2 days ago

Most of those email to sms portals no longer work, at least on the US carriers. Unless you are on a corporate account and your company pays for that additional service.

[–] warbond@lemmy.world 6 points 2 days ago

I think major American carriers stopped allowing that, actually

[–] WhyJiffie@sh.itjust.works 3 points 2 days ago

does that only exist at places where SMS is free?

[–] Norsca@lemmy.world 1 points 2 days ago

That's actually terrifying a lot of people don't even care on checking their mails on a daily basis