this post was submitted on 21 Aug 2026
38 points (95.2% liked)

GenZedong

5286 readers
48 users here now

This is a Dengist community in favor of Bashar al-Assad with no information that can lead to the arrest of Hillary Clinton, our fellow liberal and queen. This community is not ironic. We are Marxists-Leninists.

See this GitHub page for a collection of sources about socialism, imperialism, and other relevant topics.

This community is for posts about Marxism and geopolitics (including shitposts to some extent). Serious posts can be posted here or in /c/GenZhou. Reactionary or ultra-leftist cringe posts belong in /c/shitreactionariessay or /c/shitultrassay respectively.

We have a Matrix homeserver and a Matrix space. See this thread for more information. If you believe the server may be down, check the status on status.elara.ws.

Rules:

founded 5 years ago
MODERATORS
GrainEater@lemmygrad.ml
Elara@lemmygrad.ml
ComradePupIvy@lemmygrad.ml
American_Communist22@lemmygrad.ml
38
Comparison of homework and exam scores between students who use AI vs. those who don’t. (lemmygrad.ml)
submitted 1 month ago by allende2001@lemmygrad.ml to c/genzedong@lemmygrad.ml
18 comments fedilink hide all child comments
 

cross-posted from: https://hexbear.net/post/9348639

The Generative AI Learning Penalty: Evidence from Chinese Secondary Education

Using 30 months of panel data on 26,811 Chinese students in grades 7-12, we study how generative AI affects homework productivity and learning. The data combine monthly closed-book exams, high-school and college entrance exams, and homework scores and completion time across nine subjects. We exploit staggered AI adoption in a difference-in-differences design. AI adoption raises homework scores by 18% and reduces completion time by 30%, but lowers monthly exam scores by 20% within six months. High-stakes entrance-exam scores fall by 18 and 24%, with the full penalty emerging only after about two years. The losses are largest in social science subjects, followed by STEM and languages, and are especially large for junior students, high-achieving students, and boys. The learning losses are concentrated among roughly 80% of AI users whose behavior is consistent with homework outsourcing, as indicated by exceptionally short homework completion time coupled with high homework scores. AI users who maintain similar homework completion time as non-AI users experience small learning losses.

you are viewing a single comment's thread
view the rest of the comments
[–] Commiejones@lemmygrad.ml 8 points 1 month ago (1 children)

I don't know what the numbers mean. What is a "score" and why does it go up to 132? It looks like most of the data is clustered around the 80,90 co-ords and that the rest seems to just be outliers stretched out to push a naritive.

[–] AstroStelar@hexbear.net 7 points 1 month ago* (last edited 1 month ago) (1 children)

100 was set as the average score among students that don't use AI. 120 then means you scored 20% better than the control group average.

I looked through the study and the graphs are very confusing to read, but the conclusion is very concise so I'll post it here:

Conclusion and Discussion

Using large-scale evidence from a natural school setting, we find that generative AI use substantially reduces learning among secondary-school students. The central pattern is stark: AI raises homework scores and reduces homework time, but lowers performance on closed-book exams. Regular monthly exam scores fall by 20 percent, while high-stakes entrance-exam scores fall by 18 and 24 percent.

The negative learning effects fully materialize after six months for regular exam scores and after two years for entrance exams. The negative effects on learning outcomes appear to be mostly driven by the 81 percent of AI-using students, who spend less time on homework than even the fastest non-AI student, receive high homework scores matching the capability of generative AI tools they are using, and yet very low exam scores. By contrast, AI students who spend as much time on homework as non-AI students achieve similar exam scores and higher homework scores. [However, almost no AI-using students spend this much time on homework anymore after more than five months of use.]

Our findings highlight the importance of the demand side and student incentives. Much of the existing RCT literature focuses on the supply-side question of how to design AI tools that scaffold learning while limiting outsourcing. In China, as in many other countries, such tutoring tools already exist and are often available at low or zero cost. Yet most students in our setting do not choose them. Instead, they rely on general-purpose AI tools that provide quick answers and facilitate homework outsourcing.

How can we incentivize students to choose tutoring AI tools in an environment with general-purpose AI tools that give direct answers? Our findings suggest several possible interventions. First, providing credible information about the long-run learning costs of homework outsourcing may alter student behavior. Second, increasing the weight placed on closed-book, in-person assessments may help restore the connection between effort and reward. Third, parents and teachers may be more effective if they monitor inputs, such as homework time and study effort, rather than outputs such as homework scores.

I also found the section on who gets most affected interesting:

  • Effect is worse in junior high than high school, suggesting uninformed (over)use makes things worse;

  • "The average estimated effect 6-10 months after adoption for students who report using generative AI 0-1 hours per week is -5 percent, compared to -30 percent for students who report using generative AI 5 hours or more."

  • The effect is slightly worse for boys than for girls, widening the gender gap (boys on average score worse).

  • High-scoring students who adopt AI see the largest declines, poor-performing students are affected less.

[–] Rylo@lemmygrad.ml 5 points 1 month ago (1 children)

Effect is worse in junior high than high school, suggesting uninformed (over)use makes things worse;

I was always under the impression that the cognitive outsourcing is worse the earlier it starts. I am in academia, and a lot of really bright researchers, professors etc are using generative AI tools, but they have spent a good 20-30 years already reading, researching and producing high-quality output already, they are not using it to get away from thinking about hard problems, simply to speed up the process of tackling them.

It is way worse for students. I can only speak for mathematics, but the field is trying to come to gripes with what this whole ordeal will mean, and even then we are at least quite shielded from some of the problems with education (quite easy to expand and assess in-person exams, for instance).

You generally NEED a minimum of 10-20k hours working and thinking about this stuff to understand it at a high level, there are no shortcuts. Now the barrier to be able to put some slop together and pretend you've come up with a result is a lot lower, which is the shortcut many students are going for sadly.

If anyone is interested in the discussion I'll just put some good starters:

https://nitter.net/aaswaminathan01/status/2025418710198960388

https://davidbessis.substack.com/p/the-fall-of-the-theorem-economy

[–] AstroStelar@hexbear.net 8 points 1 month ago* (last edited 1 month ago)

I hold the belief that LLMs are the final nail in the coffin of an results-oriented education model that has become outmoded. I have personally experienced cognitive dissonance between my idealised notion of education and knowledge acquisition on one hand, and on the other the reality of stress, lack of reward for going beyond "good enough" and fear of my perfectionism yet slowness making me unable to catch up.

One way I keep myself from collapsing from all the things to worry about is that I try to focus on a few things I feel passionate about: the future of education is one of them.