cross-posted from: https://scribe.disroot.org/post/11979760
Google’s AI search told users that an Aachen doctor had been convicted of a sex crime. The conviction belonged to a different man, and on September 24 the Aachen Regional Court [in Germany] banned Google from repeating it, ruling that the company answers for what its AI Mode says. It is the second German court this year to treat Google’s AI answers as Google’s own words, and the first ruling to say so has now been settled rather than tested on appeal.
...
The doctor asked Google to delete the text on August 28 and followed up with a formal legal warning on September 15. Neither worked. The court’s 8th civil chamber then acted without hearing Google first, citing the urgency of false claims that were “publicly visible” to an unlimited audience. Google Ireland, which runs Google Search in the EU, now faces a fine of up to €250,000 for each breach, or detention of its managers, if it repeats three versions of the text. Google can still challenge the order.
The court’s reasoning is blunt: Google “provides AI-generated search and is therefore also responsible for the answers generated in this context.” It added that checking would have been easy: had Google “confronted its own AI” with the doctor’s objections, it would have found the statements were false.
...
That reasoning comes almost word for word from Munich. On May 28 the Munich Regional Court I ruled (case 26 O 869/26) on AI Overviews that told searchers two Munich publishing companies were “known for dubious business practices”, ran subscription traps and kept changing their names to avoid being traced. None of the linked sources said that; the first one cited did not mention the companies at all.
Google argued that it was only summarising other people’s pages, the same defence that has long protected search engines in Germany. The court disagreed. AI Overviews, it said, do not just show links but make “independent, new and substantive statements” built by evaluating and combining third-party sites, so Google must treat them as its own, including as the provider of an AI system under the EU’s AI Act. “Only it has influence over the AI,” the judges wrote.
Google said at the time it would appeal. That appeal has ended in a settlement, The Decoder reported on Saturday; the terms are not known and neither Google nor the publishers would comment. The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it.
...
It doesn't summarize, to summarize something you first need to understand what it says and even what it means.
AI in it's current form doesn't understand the meaning of anything, just the statistical chance of what the next word should be.
Al it's doing is paraphrasing, shuffling words around just enough to claim it doesn't owe the original writer anything.
Most of us learned why that was unacceptable in jr high English; it's plagiarism if done correctly and nonsense if not.
Google and other AI corps want to play it both ways, they want to reuse information for free while not being accountable for what it says. If they're not accountable because it's not their information, why aren't they paying the people who created the information?
They are taking the internet everyone has built for 30+ years, putting it in a mixer, and selling the sludge back to us.
Whether it has any sort of real understanding or not, it does summarise. It produces summaries as well as many other things. It's just very unreliable, due to a number of shortcomings that might just be fundamental: no real world model, no ability to distinguish between truth and lies, fact and fiction, except statistically, which sometimes results in nonsense. It can misrepresent, leave things out, and sometimes just make stuff up.
It does summarise, but it does it poorly and unreliably.