this post was submitted on 10 Oct 2026
509 points (93.2% liked)

Technology

88677 readers
2896 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] h0tbeef@lemmy.zip 1 points 10 hours ago* (last edited 10 hours ago) (1 children)

I don’t think that change would negatively impact the experiment, but for what they were trying to measure I’m unsure of how necessary that step would be.

They weren’t measuring how smart or good at math the participants were, they were specifically measuring changes in focus and persistence (so basically the amount of effort put forward). The findings show that people tend to just give up when you take the LLM away, and I think that with your proposed change you’d still get largely the same findings.

I would agree that LLMs aren’t the entire issue, but they are making the issue far worse at an exponential rate.

I’d be interested to read a study about what factors and/or traits predispose someone to trust an LLM, or allow themselves to become dependent on it.

[–] Rhaedas@fedia.io 1 points 10 hours ago (1 children)

They weren't measuring the math ability levels, but the changes in what those skills were per individual. But for the assisted ones, we don't know what they started with, so there's no base level to compare. If they're comparing them with the others, then that's a problem in the study since we're not measuring their math abilities.

What would a companion study reveal that does the same thing for the math part but just offers calculators instead? Probably the same? If it was dramatically different then there'd be a case that LLMs are affecting the result in some other way and it's not just the human nature of leaning on what they lost. Or what if that person had a tutor/help with them but they left for the rest of the test? Should be similar results, right?

[–] h0tbeef@lemmy.zip 1 points 7 hours ago (1 children)

What would a companion study reveal that does the same thing for the math part but just offers calculators instead?

I think that depends on the difficulty of the math in question. If it was something that most people don’t know how to do by hand (like your trigonometry example) you’d probably see a more severe drop off.

I believe they chose fractions because it’s something that everyone who’s gone through the public education system should know, and they’re more relevant to (most people’s) lives than trigonometry (people use them to split bills, calculate tip, etc.)

If you got a group of people who all should know trigonometry (due to their profession perhaps), and then ran the calculator test, it would be more analogous.

What would a companion study reveal that does the same thing for the math part but just offers calculators instead? Probably the same?

That would not be my hypothesis personally, but perhaps I’m biased, I intentionally do most of my basic math mentally and only use the calculator for more complicated equations. If I was in this study I would want to see how much of it I could do without the calculator first… perhaps for the study the calculator use would be compulsory, but I wouldn’t just give up when it was taken away if I didn’t need it in the first place.

Or what if that person had a tutor/help with them but they left for the rest of the test? Should be similar results, right?

I think that would depend on the knowledge the subject had before entering the testing scenario, and also on the quality of the tutor in question. Personally, I don’t always approach things in the conventional way, so a tutor could be more of a hindrance than a help if they disliked my methodology.

[–] Rhaedas@fedia.io 1 points 7 hours ago (1 children)

I think that would depend on the knowledge the subject had before entering the testing scenario

That was my main point on this test being an interesting question starter, but not being conclusive enough because we don't know. The subjects were purposefully random and not evaluated before the test.

fractions because it’s something that everyone who’s gone through the public education system should know

Yes, well. I don't think I need to comment on how bad the US system has become. I wonder how much higher the assisted people did while they had the help compared to others?

[–] h0tbeef@lemmy.zip 1 points 4 hours ago

I don't think I need to comment on how bad the US system has become

No, I think we’re in full agreement on that point, lol