Comment on I Went All-In on AI. The MIT Study Is Right.

• felbane@lemmy.world ⁨1⁩ ⁨day⁩ ago

  As someone who has been shoved in the direction of using AI for coding by my superiors, that’s been my experience as well. It’s fine at cranking out stackoverflow-level code regurgitation and mostly connecting things in a sane way if the concept is simple enough. The real breakthrough would be if the corrections you make would persist longer than a turn or two. As soon as your “fix-it prompt” is out of the context window, you’re effectively back to square one. If you’re expecting it to “learn” you’re gonna have a bad time. If you’re not constantly double checking its output, you’re gonna have a bad time.

  source

  • oscarjiminy@aus.social ⁨20⁩ ⁨hours⁩ ago

    @felbane @AutistoMephisto i don't have a cs degree (and am more than willing to accept the conclusions of this piece) but how is it not viable to audit code as it's produced so as it's both vetted and understood in sequence?

    source

    • felbane@lemmy.world ⁨20⁩ ⁨hours⁩ ago

      Auditing the code it produces is basically the only effective way to use coding LLMs at this point.

      You’re basically playing the role of senior dev code reviewing and editing a junior dev’s code, except in this case the junior dev randomly writes an amalgamation of mostly valid, extremely wonky, and/or complete bullshit code. It has no concept of best practices, or fitness for purpose, or anything you’d expect a junior dev to learn as they gain experience.

      Now given the above, you might ask yourself: “Self, what if I myself don’t have the skills or experience of a senior dev?” This is where vibe coding gets sketchy or downright dangerous: if you don’t notice the problems in generated code, you’re doomed to fail sooner or later. If you’re lucky, you end up having to do a big refactoring when you realize the code is brittle. If you’re unlucky, your backend is compromised and your CTO is having to decide whether to pay off the ransomware demands or just take a chance on restoring the latest backup.

      If you’re just trying to slap together a quick and dirty proof of concept or bang out a one-shot script to accomplish a task, it’s fairly useful. If you’re trying to implement anything moderately complex or that you intend to support for months/years, you’re better off just writing it yourself as you’ll end up with something stylistically cohesive and more easily maintainable.

      source

      • oscarjiminy@aus.social ⁨19⁩ ⁨hours⁩ ago

        @felbane thanks, such a thorough response really appreciate your time.

        source