this post was submitted on 07 Jun 2026
595 points (98.7% liked)
Technology
85212 readers
4072 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
What is the actual “cost” after they buy the hardware, is that $1000 really pure power usage cost?
The problem is that the hardware has a 5 or 6 year depreciation schedule on paper, but NVIDIA keeps saying that their next generation chip will be twice as good as their last chip so there is a FOMO schedule of like every two years.
Would be nice to see that used hardware for sale rather than it being junked as a writeoff.
that's the $84,000 question. They're filling datacenters with the fastest possible equipment and need it to be 10x faster, That hardware is dinosaur fodder a year after they install it.
I'm curious as well. My knowledge is probably quite outdated, but from what I understood the training part is what's expensive and then querying the model is pretty cheap. Is it still true (or was it ever) that the generated answers on search engines are cheaper to generate than the actual search results?
I find that hard to believe, I recently had to uninstall co-pilot after it weaseled its way into my search bar. Its not an exageration to say that my PC literally ran cyberpunk 2077 with pathtracting better than it ran the fucking windows search bar with co-pilot.
That's just a shitty front end interface implementation, it has nothing to do with the actual inference run by the models.
Look at the public numbers, it seems true. Copilot on your taskbar is just windows being garbage, not the AI being bad. Just look at self-hosted AI and measure the power costs of your queries. It’s tiny.
It is sorta. Training is orders of magnitudes more intensive than inference, but we infer billions of times within a model generation.