Comment

Comment on Microsoft Confirms Windows 11 Bug That Locks Users Out of the C: Drive

You know what’s going on inside the large companies that are hoping to cash in on the AI thing? All workers are being pushed to use AI and goals are set that targets x% of all code written be AI-generated.

And AI agents are deceptively bad at what they do. They are like the djinn: they will grant the word of your request but not the spirit. Eg they love to use helper functions but won’t necessarily reuse helper functions instead of writing new copies each time it needs one.

Here’s a test that will show that, with all the fancy advancements they’ve made, they are still just advanced text predictors: pick a task and have an AI start that task and then develop it over several prompts, test and debug it (debug via LLM still). Now ask the LLM to analyse the code it just generated. It will have a lot of notes.

An entity using intelligence would use the same approach to write the code as it does to analyze it. Not so for an LLM, which is just predicting tokens with a giant context window. There is no thought pattern behind it, even when it predicts a “thinking process” before it can act. It just fits your prompt into the best fit out of all the public git depots it was trained on, from commit notes and diffs, bug reports and discussions, stack exchange exchanges, and the like, which I’d argue is all biased towards amateur and beginner programming rather than expert-level. Plus it includes other AI-generated code now.

So yeah, MS did introduce bugs in the past, even some pretty big ones (it was my original reason for holding back on updates, at least until the enshitification really kicked in), but now they are pushing what is pretty much a subtle bug generator on the whole company so it’s going to get worse, but admitting it has fundamental problems will pop the AI bubble, so instead they keep trying to fix it with bandaids in the hopes that it’ll run out of problems before people decide to stop feeding it money (which still isn’t enough, but at least there is revenue).

source

Sort:hotnew top

ExperiencedWinter@lemmy.world ⁨8⁩ ⁨hours⁩ ago

Now ask the LLM to analyse the code it just generated. It will have a lot of notes.

Not only will it have a lot of notes, every time you ask if to analyze the code it will find new notes. Real engineers are telling me this is a good code review tool but it can’t even find the same issues reliably. I don’t understand how adding a bunch of non-deterministic tooling is supposed to make my code better.

source
- Buddahriffic@lemmy.world ⁨8⁩ ⁨hours⁩ ago
  Though on that note, I don’t think having an LLM review your code is useless, but if it’s code that you care about, read the response and think about it to see if you agree. Sometimes it has useful pointers, sometimes it is full of shit.
  
  source
  - ExperiencedWinter@lemmy.world ⁨7⁩ ⁨hours⁩ ago
    So when do I stop asking the LLM to take another look? If it finds a new issue on the second or third or fourth check am I supposed to just sit here and keep asking it to “pretty please take another look and don’t miss anything this time”?
    
    I’m not saying it’s a useless tool, it’s just not a replacement for a human code review at all.
    
    source
    Buddahriffic@lemmy.world ⁨7⁩ ⁨hours⁩ ago
    Stop when you feel like it, just like any other verification method. You don’t really prove that there are no problems with software development, it’s more of a “try to think of any problem you can and do your best to make sure it doesn’t have any of those problems” plus “just run it a lot and fix any problems that come up”.
    
    An LLM is just another approach to finding potential problems. And it will eventually say everything looks good, though not because everything is good but because that happens in its training data and eventually that will become the best correlated tokens (assuming it doesn’t get stuck flipping between two or more sides of a debated issue).
    
    source
  - JcbAzPx@lemmy.world ⁨8⁩ ⁨hours⁩ ago
    That sounds worse than useless. It would be better to fail utterly than make up shit that you have to waste time parsing through.
    
    source
    Buddahriffic@lemmy.world ⁨7⁩ ⁨hours⁩ ago
    It helps in the sense of once you’ve looked at code enough times, you can stop really seeing it. So many times I’ve debugged issues where I looked many times at an error that is obvious in hindsight but I just couldn’t see it before that. And that’s in cases where I knew there was an issue somewhere in the code.
    
    Or for optimization advice, if you have a good idea of how efficiency works, it’s usually not difficult to filter the ideas it gives you into “worthwhile”, “worth investigating”, “probably won’t help anything”, and “will make things worse”.
    
    It’s like a brainstorming buddy. And just like with your own ideas, you need to evaluate them or at least remember to test to see if it actually does work better than what was there before.
    
    source
SoleInvictus@lemmy.blahaj.zone ⁨1⁩ ⁨day⁩ ago
You’re spot on regarding how AI operates.

AI is stupid story time!

I recently helped a friend with a self-hosted VPN problem. He had been using a free trial of Gemini Pro to try to fix it himself but gave up after THREE HOURS. It never tried to help him diagnose the issue, but instead kept coming up with elaborate fixes with names that suggested they were known issues, like The MTU Traffic Jam, The Packet Collision Quandary, and, my favorite, The Alpine Ridge Controller Trap. Then it would run him through an equally elaborate “fix”. When that didn’t work, it would use the failure conditions to propose a new, very serious sounding pile of bullshit and the process would repeat.

I fixed it in about fifteen minutes, most of that time spent undoing all the unnecessary static routing, port forwarding, and driver rollbacks it had him do. The solution? He had a typo in the port number in his peer config.

I can’t deny that LLMs are full of useful knowledge. I read through its output and all of its suggestions absolutely would have quickly and efficiently fixed their accompanying issue, even the thunderbolt/pcie bridging issue, if the real problem had been any of them. They’re just garbage at applying that information.

source
- Buddahriffic@lemmy.world ⁨8⁩ ⁨hours⁩ ago
  Yeah, they don’t do analysis but can fool people because they can regurgitate someone else’s analysis from their training data.
  
  If could just be matching a pattern like “I have a network problem with <symptoms>. Your issue is <problem> and you need to <solution>.” Where the problem and solution are related to each other but the problem isn’t related to the symptoms, because the correlation with “network problem” ends up being stronger than the correlation with the description of the symptoms.
  
  And that specific problem could likely be solved just by adding a description of that process to the training data. But there will be endless different versions of it that won’t be fixed by that bandaid.
  
  source