Comment on OpenAI Will Shut Down Sora Video Platform

• StupidBrotherInLaw@lemmy.world ⁨1⁩ ⁨week⁩ ago

  Unpopular Opinion Incoming

  I was assigned at work to evaluate a few LLMs for potential adoption, so I spent a solid week doing so.

  Most of the “AI is broken and doesn’t work” on here is solid echo chamber cope. It’s more competent than several of my coworkers, though it’s thankfully not ready to replace knowledge workers as it requires a knowledge baseline to best direct it and evaluate its answers.

  I still advised against using it for multiple reasons, including ethics, but much of Lemmy is playing make believe about the actual capabilities of LLMs.

  source

  • queermunist@lemmy.ml ⁨1⁩ ⁨week⁩ ago

    Cool anecdote. Every time we actually see real data, though, the numbers don’t reflect any productivity gains or increased efficiency or better output. People say that LLMs are useful because it feels useful, but we aren’t seeing actual usefulness. The most recent study out of Duke University observes “a productivity paradox, in which perceived productivity gains are larger than measured productivity gains, likely reflecting a delay in revenue realizations.”

    A delay. Sure.

    source

    • StupidBrotherInLaw@lemmy.world ⁨1⁩ ⁨week⁩ ago

      I really appreciate your dismissive, arrogant tone. It really added to the comment. Thank you!

      source

      • queermunist@lemmy.ml ⁨1⁩ ⁨week⁩ ago

        I directly quoted a study from Duke University, how is that “even less substance” that your anecdote?

        source
  • Erdalion@lemmy.world ⁨1⁩ ⁨week⁩ ago

    Mind telling us what it is that you do? I heard similar things being said in the Plain English podcast last week (and the host was pretty anti-AI before) and I’m starting to wonder if certain jobs are going to be more affected than others.

    Or are your coworkers just bad at what they do? :P When I was working tech support, there were people that were worse at their jobs than the bots of the time, let alone LLMs, I swear.

    source

    • StupidBrotherInLaw@lemmy.world ⁨1⁩ ⁨week⁩ ago

      Electrical engineering. My mentioned coworkers are competent but more junior in the field. We did a miniature internal study and found the best models provided accurate, relevant information on the first prompt about 90% of the time when asked to explain or verify concepts. The remainder consisted of hallucinations or misunderstood queries.

      They struggled with questions that instead required complex problem, providing some mixture of appropriate solutions, overly complex but still functional solutions, and hallucinated shite.

      I recommended that we do not move forward with adopting AI in any capacity. While it has some utility for basic information retrieval and fact checking, it still required someone with sufficient knowledge to be able to quickly evaluate the quality of its output. Helpful for someone who knows what they’re doing, dangerous 10% if the time for someone who does not. I also highlighted the ethical concerns, many of which my peers were unaware.

      source