511
Microsoft executive vice president bemoans employees sending him "AI slop," saying "This is a doom loop."
(www.windowscentral.com)
We're not The Onion! Not affiliated with them in any way! Not operated by them in any way! All the news here is real!
Posts must be:
Please also avoid duplicates.
Comments and post content must abide by the server rules for Lemmy.world and generally abstain from trollish, bigoted, ableist, or otherwise disruptive behavior that makes this community less fun for everyone.
And that’s basically it!
Microsoft (and maybe all the AI companies) made at least few mistakes in the marketing of this technology. AI isn't a replacement for human accountability. It still requires oversight and expertise. It is great for a first draft. It is terrible for a complete work product. Agents (with the right loops to verify accuracy) are reasonably complex - and most companies don't have the people development infrastructure to train their teams with the right skills to build them. Honestly, AI is shining a light on how terribly businesses have been run over the last 30 years, and how lazy business leaders are.
It is terrible for a first draft if you're aiming for something that reads like a human being's actual thoughts. Don't let an LLM think for you and create the first draft of "your" thoughts. At most, if you absolutely must, use it as a spell checker. But write your words yourself.
No one wants to know what ChatGPT thinks. We might want to know what you think... But only if it's actually you doing the thinking.
Sorry but I don't need my thoughts to write a cover letter lol
Nor do you need an LLM - someone makes a cover letter once and then you use it as a template.
Only if the aim is towards mediocrity.
Do you aim for excellence in your first drafts? You may want to consider better effort allocation.
If you are using a generator that combines everything (both bad and good) on the net, you will end right on the middle. If everyone that's lazy uses the same method, your work becomes the definition of mediocre.
Nothing wrong with mediocre, I mean the corporate world is full of mediocrity and that's what is expected in assignments.
That assumes that there is no effective way to filter good from bad. But there is - both automated heuristics and manual training does this.
LLMs absolutely produce mediocre output in some ways, but it's not an inherent limitation caused by them "averaging" the internet. If that were the case there'd be a lot more typos, emojis and internet lingo by default. The fact that LLMs have these instantly recognisable stock ways of writing and stock phrases is a simple way of seeing that they don't simply produce "average" output in that very naive sense.
The fact that they "average" their inputs is why there are comparatively few typos (different sources have different typos, so they average out), not too many emojis or internet lingo (again, different sources use different ones in different places, so they average away), and why they produce such tedious stock output (it's an average of the inputs, so all the little quirks and idioms that make human communucation more vibrant have been blended away).
I'm sure there is some filtering on the inputs to try to remove the worst of it, but ultimately it's still just taking the rest and building it's probability tables from that, which leads to the homogenised outputs we see.
Mind you, having said there are fewer typos, the last time I bothered trying to get one to write some code, it managed to misspell a popular library name in multiple places, which gives some indication of how bad the inputs are, how bad the tokeniser is, or possibly both.
If "different typos" averaged out to "nearly no typos" the same logic would have different words average out to nearly no words. What actually happens is the model learns context, and can produce output which contains emojis in one context and not others. These contexts can be very far from the average context.
I'm afraid the upshot is you don't understand how the models work. There is extensive filtering before training - they do not get "the entire internet" and average it. If you want to understand properly, there are a lot of resources that will let you, but I'm not going to try to do it here, so you'll either have to believe me or be wrong, I'm afraid.
No, because typos are irregular, so combining multiple sources does not reinforce them, whereas "words" (tokens would be a better term, because they're not always full words) tend to be used in similar ways, reinforcing those patterns. As you say, context is relevant, an LLM isn't just looking at the last token to decide the next, but at a much larger window. That does allow it to adjust to tone, as the probabilities of certain tokens, and so words, will depend on that tone, and the type of words used, and thus context, of a conversation. If emojis are used a lot in certain contexts, those patterns will tend to be reinforced in their training, and so produced more in their output.
As to filtering their input, at no point did I say they ingest "the entire internet", so quoting it seems rather disingenuous. They scrape as much text as they can get, both online, and by OCRing books, as we've seen with the recent upset about the number they destroy. What the commercial models do with this afterwards is uncertain, as anything they say is likely to be misleading for commercial purposes. I think it's a fair assumption that they want good quality data, however they define that, but filtering it all manually is obviously much too vast a project to do entirely manually, so it's done heuristically, which has the obvious problem that it'll let through low quality sources some of the time, lowering the quality of the overall data set. You only need to read the anodyne screed they produce to see how all of the little quirks and nuance that marks human communication tends to get left out, leaving LLM prose feeling rather vacuous and repetitive.
I was going off the statement, "If you are using a generator that combines everything (both bad and good) on the net". I think "the entire internet" is an OK paraphrase of "everything on the net", but maybe you meant something else than what I understood.
The original public release of ChatGPT used a small army of humans to generate and curate training data, and it's still the rule today. So what the average of what the model actually sees is quite far from the average of everything on the net.
The ability of the model to learn different contexts and tones means that all it takes to not land "right on the middle" is a slightly different context, which can all be contained in the instructions given to the model. Sure, the models have their stupid phrasings and stock phrases, but the context was producing a first draft. A competent human can reword those very easily.
Do you even know what the word heuristic means? A heuristic is something that is just good enough. Not great, not perfect, just good enough to get the job done.
Heuristic algorithms were always going to result in LLMs that were only just good enough.
Heuristics are by definition imperfect, but they are not, generally, "just good enough". In fact, heuristics may not be good enough for a given purpose.
Am I right that you're not actually disagreeing with my comment?
Heuristics that aren’t good enough aren’t used for that purpose... what kind of rebuttal is this even?
It's not a rebuttal because I don't even know what point you're trying to make, as you may have been able to tell from my question:
As my violin teacher used to say, “practice doesn’t make perfect, perfect practice makes perfect.”
If you’re not aiming for excellence every time you do something you’re practicing imperfection.
Sure man, but the vast majority of us do not work in the making beautiful sounds with violing business.
If some middle manager boss says to some minion to make a quick draft of a keynote of some random info that tangentially may make sense in one of his meetings, and the minion then goes on working on it on full attention for a day to make the beautifullest keynote of the word he did not do a draft and the boss is pissed because other shit did not get done. Having an llm quarter ass a keynote based on a prompt and two refinements is precisely the effort required for that job, any more than that is wasting time.
If the quick draft doesn't justify the time and energy, the keynote certainly doesn't. If the keynote is a waste of time, then the meeting is pointless. The best place for the LLM is to replace the jackass who called the pointless meeting in the first place.
This thread of comments reads like you just now discovered bullshit corporate jobs exists. Welcome to 1985. Some books to help you navigate are: Dealing with people you can not stand, bullshit jobs, almost any work stress self help book, and arguably American Psycho - specifically the movie version but the book is fine. Embrace the 80/20 rule, don't let perfect be the enemy of good, show solidarity with your fellow workers, and just play the game until you can reach a point to enact change.
I agree with you. Yet those jackasses will not be replaced any faster by us agreeing here.
Perhaps some people who don't mind the ethics of gen AI work very hard to produce good work. But I believe that when you give people an easy, low-effort path, a great many of them will take it (regardless of the quality of the results). Training notwithstanding, a vast number of people cannot resist the lure of doing as little as possible.
Yeah I get the feeling lazy people love it, and people who work hard know it doesn't produce very good results in most circumstances.
Unfortunately it seems a lot of managers are in the lazy group. So they assume that when employees tell them it doesn't work very well, they think it's because they have a "bad attitude" or something like that.
First draft of what? The AI* doesn't know what you're trying to make, so to use it, you first need to write a prompt to convey to it what you're trying to convey. If what you're trying to make takes the form of prose, there's your first draft already; there's nothing the AI can add other than padding and replacing your voice with a psychopath's. But to begin with, if you have the skill to turn a first draft into a final product, it's almost always going to be faster to just use that skill to create the first draft yourself instead of trying to convey to an AI what it's supposed to be like. If it isn't, it's something that's so easy to describe that there's either no value in making it or those few words are already the perfect way to convey it.
Also, this just completely undermines the whole thing:
The sole virtue AI arguably has, is its accessibility; anyone who can read and write a supported natural language, can make use of it. But the moment it starts requiring expertise, that accessibility becomes worthless. To anyone who is even somewhat serious about what they're trying to do, that natural language interface just offers far too little control for their purposes.
*) For the purposes of this comment, "AI" refers to the kinds of natural language-driven generative AI heavily marketed by companies like Microsoft, not the general concept of artificial intelligence or even the underlying technology of those products.
This is an incredibly stupid take, can’t believe people upvoted this shit, lol
Have you not used AI agents anytime in the past 6 months?
They can pull in information that you don’t write down, it can create Powerpoint slides, it can create mermaid diagrams from your description.
Those are not things you just do manually because you can? At that point why not code in Notepad instead of relying on the stupid machine assisted IDEs?
Any information that whatever you're working on hinges upon, you need to already know and have confirmed before you start working on it. So not only did you need to look it up anyway, only you know what aspects of that information are most important for the goals of your work. Of course, there can also be less vital pieces of information that you may consult as you work, like maybe you want to compare the size of something with football fields, to make it harder to understand. Undoubtedly an AI can do that for you, but in the end you still need to look it up to verify it, so you haven't really saved any time or effort.
It's a similar kind of story for things like generating PowerPoint slides. Slides based on what? An article? The script? Slides that just summarise the prose make for a terrible presentation and aren't any better as a starting point for one. Also, anything that can be described using less time and (cognitive) energy than to just write it in Mermaid code, isn't worth making a diagram out of.
This is comparison makes no sense, because an IDE is the very opposite of AI. There is little value in a person manually performing the same algorithms over and over again, other than maybe for a learning process. So if they can be automated, it just means that people can focus their efforts on doing the work that requires, you know, personhood. AI is the opposite in that it's useless for procedural automation, and the only thing it can do automatically is the kind of work that can only have value if a person is doing it.
Ok
And it is usually half right, its really a mess, unless you give it a really good rough draft outline.
I find writing the code is the easy part. The difficult part is figuring out what the requirements are, fitting together all of the data relationships, figuring out the security so the correct people and systems have access to the data. To do these things I need very specific knowledge of the systems I'm working on, knowledge the AI doesn't have.
I could see it could give you a simple interface that's similar to something it's been trained on. But it's capabilities are very limited in terms of doing anything novel. It can't actually understand a problem, it can only give some code similar to problems that exist in it's training data. Sometimes that's useful, since at times I do have to solve problems that's been solved by others. But most of the time the AI can't understand what I'm trying to do and it's easier to write the code than it is to try to explain to an LLM how to write the code I want it to write.
Maybe you are doing some exotic stuff, but for me it’s more than capable of writing code, coding is mostly about applying existing patterns to solve known problems, very few programmers deal with novel problems.
Design Patterns book has been out for what? 2 decades, and it’s not outdated.
For good team interoperability you want to have code that’s standardized, that’s also where LLMs excel at.
It’s a nice productivity tool, just like an IDE is a nice productivity tool
In software engineering, sure there's patterns, but that's just guidelines. Eventually you run up against a problem where an something you don't have control over isn't 100% reliable so you have to think through how to handle that scenario. Maybe you retry an API call, maybe you notify some people about the issue, it's all context dependent what you need to do. The AI doesn't know what to do, because how would it? There are a massive (I won't say infinite, but in terms of a human lifespan it may as well be) number of combinations of technology X connecting to technology Y. So there is always going to be a great many problems that no one has ever encountered before. LLMs increasing the number of features in the software doesn't solve those kinds of problems it just increases the number of problem.
I think it's fine to think of it as a tool, like an IDE. But just like an IDE, it doesn't solve the problems, the person using the tool solves the problem.
For instance today, the LLM explained to me how to enable SSL certificate negotiation with TLS 1.3 some shitty IIS setup I have to maintain. I used the LLM because I figured that I'm probably not the only one that has seen that issue. Now the problem I have to deal with is whether the software that updates the certs might break the config, that's a little less likely the LLM will have a reliable solution to. I might just wait until the software updates the cert an manually test if it broke. The day before I had to launch some stuff in the middle of the night so I wouldn't interfere with business operations. The launch was to change something from single linked list to a doubly linked list which is important for reasons I can't explain now. I mean I would explain it, but honestly I can't remember why. I just have in my notes that it needs to be a doubly linked list for the recursion to work properly. Maybe next week when I fix the FE I'll remember why, but if I was correct many months ago (it's a low priority issue), the FE should now have the data it needs to solve whatever problem was there.
So yeah, I use LLMs sometimes for problems that I think other people have seen before but sure as hell can't be driving on what I do. Someone has to make the various pieces of software work together, know the business the software is being used in. More software means more problems that someone has to figure out. If you make n pieces of software with LLMs, you're make n! problems to solve.
Yes AI or LLM isn’t able to code solution itself, but it can fill in a lot of gaps based on what you tell it, it can copy existing patterns.
And you can direct it, it’s a tool like an IDE, just much better .
Attention driven economy. All enterprises now behave like they had a field week at used cars salesmen retreat. You can't sell unless the value proposition is grotesquely inflated.