Comment on UK government trial of Microsoft's M365 Copilot finds no clear productivity boost

gravitas_deficiency@sh.itjust.works ⁨1⁩ ⁨day⁩ ago

• ExcessShiv@lemmy.dbzer0.com ⁨1⁩ ⁨day⁩ ago

  According to the M365 Copilot monitoring dashboard made available in the trial, an average of 72 M365 Copilot actions were taken per user.

  “Based on there being 63 working days during the pilot, this is an average of 1.14 M365 Copilot actions taken per user per day,” the study says. Word, Teams, and Outlook were the most used, and Loop and OneNote usage rates were described as “very low,” less than 1 percent and 3 percent per day, respectively.

  Yeah that probably won’t have the intended effect…this basically just shows that AI assistants provide no benefit when they’re not used and nothing else.

  source

  • cmnybo@discuss.tchncs.de ⁨1⁩ ⁨day⁩ ago

    People probably tried it, found out that it’s crap and stopped using it.

    source

    • ExcessShiv@lemmy.dbzer0.com ⁨1⁩ ⁨day⁩ ago

      Its hardly possible to actually test it properly in relation to your work and changes in productivity with a single query per day. It

      source

      • 31ank@ani.social ⁨1⁩ ⁨day⁩ ago

        They probably did multiple queries per day at the beginning, found out it isn’t worth it and stopped using it …

        source

        • -> View More Comments
      • tartarin@reddthat.com ⁨1⁩ ⁨day⁩ ago

        You assume the average usage is representative of the actual usage. You averaged the actions over the time period, nothing’s says the users didn’t performed the averaged 72 actions within the first three days or any time restricted window within the whole period of time and got bored with it seeing no or low value.

        source

        • -> View More Comments
      • echodot@feddit.uk ⁨1⁩ ⁨day⁩ ago

        I’m not a programmer, so it’s got nothing tonoffer me. Mostly my job is to write documentation for propriety software and hardware, stuff the AI knows nothing about, not everyone in the world can maker use of AI and it doesn’t require a PhD and 30!days of constant usage to work that out.

        source

        • -> View More Comments
  • Jhex@lemmy.world ⁨1⁩ ⁨day⁩ ago

    .this basically just shows that AI assistants provide no benefit when they’re not used and nothing else.

    so you think they may be useful but people just like to work harder? or perhps, they tried and saw no benefit at all and moved on?

    source

    • ExcessShiv@lemmy.dbzer0.com ⁨1⁩ ⁨day⁩ ago

      Having been part of multiple projects introducing new software tools (not AI) to departments before, people are usually just stubborn and don’t want to change their ways, even if it enables a smoother work-flow with minimal training/practice.

      source

      • Jhex@lemmy.world ⁨1⁩ ⁨day⁩ ago

        The devil is in the details… what you describe screams to me what I call the “new boss syndrome”. New boss comes in and they feel the need to pee on everyone to mark their territory so they MUST bring in some genius change.

        99% of the time, they are bringing in some forced change for the sake of change or something that worked on their previous place without taking into consideration the context.

        I do not know almost anyone who prefers to work harder… either the changes proposed make no sense (or it’s too complex for people to understand the benefit) or the change is superfluous. That is usually where resistance to change comes from.

        source
      • rebelsimile@sh.itjust.works ⁨1⁩ ⁨day⁩ ago

        In all your software deployments did you blame the users for not getting it or did you redesign the software because it sucked (according to your users)?

        source

        • -> View More Comments
  • echodot@feddit.uk ⁨1⁩ ⁨day⁩ ago

    We have it on our system at work. When we asked what management expected it to be used for they didn’t have an answer.

    We have a shell script that ingests a list of user IDs and resets their active directory passwords, then locks the account, then sends them an email telling them to contact the support desk to unlock the account. It a cron job that runs ever Monday morning.

    What do a need an AI for when we can just use that? A script that can be easily read understood and upgraded, new concerns about it going off-piste and doing something random and unpredictable.

    source

    • sugar_in_your_tea@sh.itjust.works ⁨1⁩ ⁨day⁩ ago

      Well yeah, AI shouldn’t replace existing, working solutions, it should be used in the research phase for new solutions as a companion to existing tools.

      source
  • panda_abyss@lemmy.ca ⁨1⁩ ⁨day⁩ ago

    Worth noting the average includes the people who did use it a lot too.

    So you can conclude people basically did not use it at all.

    source