UK government trial of Microsoft's M365 Copilot finds no clear productivity boost

Comments

• gravitas_deficiency@sh.itjust.works ⁨1⁩ ⁨day⁩ ago

  Kinda want to send this to my company lol

  source

  • ExcessShiv@lemmy.dbzer0.com ⁨1⁩ ⁨day⁩ ago

    According to the M365 Copilot monitoring dashboard made available in the trial, an average of 72 M365 Copilot actions were taken per user.

    “Based on there being 63 working days during the pilot, this is an average of 1.14 M365 Copilot actions taken per user per day,” the study says. Word, Teams, and Outlook were the most used, and Loop and OneNote usage rates were described as “very low,” less than 1 percent and 3 percent per day, respectively.

    Yeah that probably won’t have the intended effect…this basically just shows that AI assistants provide no benefit when they’re not used and nothing else.

    source

    • cmnybo@discuss.tchncs.de ⁨1⁩ ⁨day⁩ ago

      People probably tried it, found out that it’s crap and stopped using it.

      source

      • -> View More Comments
    • Jhex@lemmy.world ⁨20⁩ ⁨hours⁩ ago

      .this basically just shows that AI assistants provide no benefit when they’re not used and nothing else.

      so you think they may be useful but people just like to work harder? or perhps, they tried and saw no benefit at all and moved on?

      source

      • -> View More Comments
    • echodot@feddit.uk ⁨1⁩ ⁨day⁩ ago

      We have it on our system at work. When we asked what management expected it to be used for they didn’t have an answer.

      We have a shell script that ingests a list of user IDs and resets their active directory passwords, then locks the account, then sends them an email telling them to contact the support desk to unlock the account. It a cron job that runs ever Monday morning.

      What do a need an AI for when we can just use that? A script that can be easily read understood and upgraded, new concerns about it going off-piste and doing something random and unpredictable.

      source

      • -> View More Comments
    • panda_abyss@lemmy.ca ⁨21⁩ ⁨hours⁩ ago

      Worth noting the average includes the people who did use it a lot too.

      So you can conclude people basically did not use it at all.

      source
• octopus_ink@slrpnk.net ⁨1⁩ ⁨day⁩ ago

  I love that the only thing the oligarchy can focus on is making sure we can all use AI to work more.

  Image

  source

  • PrettyFlyForAFatGuy@feddit.uk ⁨21⁩ ⁨hours⁩ ago

    If you can be in three meetings at once with AI then every single one of those meetings could have been an email

    source

    • frezik@lemmy.blahaj.zone ⁨19⁩ ⁨hours⁩ ago

      There’s meetings other people need to have and I just need to know broadly what was said. Transcription and summerizing would be great for that

      That is, if I could trust its accuracy. Which I don’t.

      source

      • -> View More Comments
  • abbiistabbii@lemmy.blahaj.zone ⁨1⁩ ⁨day⁩ ago

    I mean basically. Call me a paranoid communist but given half the chance they’d fucking bring back slavery.

    source

    • SpicyLizards@reddthat.com ⁨22⁩ ⁨hours⁩ ago

      I think we are there, just under the name of capitalism vs slavery.

      source
    • mrgoosmoos@lemmy.ca ⁨17⁩ ⁨hours⁩ ago

      that’s pretty much where we are now

      shit minimum wage, corporations owning housing, and monopolies in pretty much every market. it’s just slavery with the illusion of freedom because you can choose which shitty apartment building to live in for over half your income, and which franchise stores you shop at, while your essentials are getting price gouged and constantly worse quality for higher cost, yet the workers don’t make more

      that’s just slavery with extra steps

      source
    • vacuumflower@lemmy.sdf.org ⁨21⁩ ⁨hours⁩ ago

      There are bad things in communism - a reductionist model advertised as fitting for everything (a bit similar to Unix, that), and there are good things in communism - attention to balance of power. Revolutionary ideologies generally have situations warranting a revolution more fleshed out.

      source

      • -> View More Comments
  • SpicyLizards@reddthat.com ⁨22⁩ ⁨hours⁩ ago

    She looks happy too!

    source

    • jubilationtcornpone@sh.itjust.works ⁨17⁩ ⁨hours⁩ ago

      I believe that’s the “I spent six years in college and $150,000 for the ‘privilege’ of sitting in teams meetings all day.” look.

      source
    • octopus_ink@slrpnk.net ⁨22⁩ ⁨hours⁩ ago

      I like to imagine we are witnessing malicious compliance from the model.

      source
• cerebralhawks@lemmy.dbzer0.com ⁨22⁩ ⁨hours⁩ ago

  Yeah, no shit. But they nearly doubled the price. I canceled my membership, but I doubt enough did to actually matter.

  I was fine paying $60 a year for Office. I was never gonna use the AI stuff. When they said it was $100, I bailed. So now they don’t get the $60. But enough people will go on paying that they will actually make more money on Office in the next year, not less.

  Not enough people are willing to vote with their wallets or even their feet to effect any meaningful change. At least not when it comes to their tech toys.

  source

  • ArmchairAce1944@discuss.online ⁨14⁩ ⁨hours⁩ ago

    I have been using Libre office for a while now and it’s superior to office in every way.

    source
  • FlashMobOfOne@lemmy.world ⁨18⁩ ⁨hours⁩ ago

    Not enough people are willing to vote with their wallets

    That and most governments are wrapped up in Windows, and therefore kinda just captive to the insane pricing. I get everything I need out of LibreOffice, personally.

    source
  • jubilationtcornpone@sh.itjust.works ⁨17⁩ ⁨hours⁩ ago

    The sole reason I still pay the Microsoft tax is Excel. Other office suite components are generally good enough to fill in for their Microsoft counterparts. But, spreadsheet programs are one area where open source competitors need to get their shit together.

    Most of them can do the basics but Excel is still in a class by itself for power users and advanced functionality. That’s a real bummer because I would love to stop paying the Microsoft tax.

    source

    • Illecors@lemmy.cafe ⁨16⁩ ⁨hours⁩ ago

      I’m no dev, but would you consider writing up in detail the features/behaviour you’re missing on libreoffice issue tracker?

      source
• tekato@lemmy.world ⁨17⁩ ⁨hours⁩ ago

  I don’t see where a government would need a chatbot. Anyways, chances are that half the staff was already using some form of LLM before this trial.

  source

  • sugar_in_your_tea@sh.itjust.works ⁨16⁩ ⁨hours⁩ ago

    Why wouldn’t they want one? If it’s a tool their employees want, they should provide it.

    source

    • Treczoks@lemmy.world ⁨16⁩ ⁨hours⁩ ago

      The point is that this is all happening in a cloud. One that is probably located in the US. Not a good thing for a non-US government to send potentially confidential or even secret data to.

      source

      • -> View More Comments
• SpicyLizards@reddthat.com ⁨23⁩ ⁨hours⁩ ago

  No shit ‐ the AI bubble provides no value, but it is exciting for the c suite and governments.

  source
• Katana314@lemmy.world ⁨14⁩ ⁨hours⁩ ago

  Ugh, thought this could’ve referred to a Trial as in “All rise for the judge”, not Trial as in “Your free trial has expired”.

  We’re way overdue to put AIs on former trials.

  source
• prole@lemmy.blahaj.zone ⁨20⁩ ⁨hours⁩ ago

  Lots of LLM shills in these comments. I hope your work doesn’t value reality/accuracy.

  source

  • Animated_beans@lemmy.world ⁨15⁩ ⁨hours⁩ ago

    I use Copilot for generating images of concepts for presentations at work. It helps me get my point across and no accuracy is needed because it is taking the place of clip art and Google image searches. There is absolutely a place for Generative AI in the workplace. Whether it is worth the cost and whether people are trusting it too much is another question.

    source
  • Lemminary@lemmy.world ⁨13⁩ ⁨hours⁩ ago

    It helps me get there more often than not, anywhere from programming I’m unfamiliar with to brainstorming in graphic design. I see a lot of anti-AI folks diss it without considering how it’s actually used. It’s a tool like any other, and you get what you make of it.

    source
• Hadriscus@jlai.lu ⁨19⁩ ⁨hours⁩ ago

  They just want to get rid of the working class for good. Let’s cut their heads instead ❤️

  source

  • sugar_in_your_tea@sh.itjust.works ⁨16⁩ ⁨hours⁩ ago

    If they do, who would buy their stuff?

    source

    • e461h@sh.itjust.works ⁨16⁩ ⁨hours⁩ ago

      They really don’t think past the next quarter or two. And even if they did they aren’t lacking on new ways to exploit people.

      source
• AceBonobo@lemmy.world ⁨1⁩ ⁨day⁩ ago

  From reading the study, it seems like the workers didn’t even use it. Less than 2 queries per day? A third of participants used it once per week?

  This is a study of resistance to change or of malicious compliance. Or maybe it’s a study of how people react when you’re obviously trying to take their jobs.

  source

  • echodot@feddit.uk ⁨1⁩ ⁨day⁩ ago

    I don’t think it’s people being resistant to change I think it’s people understanding the technology isn’t useful. The tagline explains it best.

    AI tech shows promise writing emails or summarizing meetings. Don’t bother with anything more complex

    It’s a gimmick, not a fully fleshed out productivity tool, of course no one uses it. That’s like complaining that no one uses MS paint for the production of a high quality graphics.

    source

    • SpicyLizards@reddthat.com ⁨22⁩ ⁨hours⁩ ago

      Absolutely, and it’s a massive and undeserved cash cow for AI companies (e.g. Sam Sister Altman).

      AI is never an investment for businesses or individual users. It’s a bloated and unfulfillable promise that just makes users dumb, dependant, and destroys the very environment we need to survive.

      It also produces bad products (it’s easy to tell which devs use it from reviewing poor quality code).

      Not to mention the centralisation of power with the rich who are the problem in this world.

      source
  • thehatfox@lemmy.world ⁨1⁩ ⁨day⁩ ago

    The figures are the averages for the full trial period.

    So it’s possible they were making more queries at the start of the trial, but then mostly stopped when if they found using Copilot was more a hindrance than a help.

    source

    • elvith@feddit.org ⁨23⁩ ⁨hours⁩ ago

      I have a Copilot license at work. We also have an in house „ChatGPT clone“ - basically a private deployment of that model so that (hopefully) no input data gets used to train the models.

      There are some usecases that are neat. E.g. we’re a multilingual team, so having it transcribe, translate (and summarize) a meeting so that it’s easier to finalize and check a protocol. Coming back from a vacation and just ask it summarize everything you missed for a specific area of your work (to get on track before just checking everything chronologically) can be nice, too.

      Also we finetuned a model to assist us in writing and explaining code from a domain specific language with many strange quirks that we use for a tool and that has poor support from off the shelf LLMs.

      But all of these cases have one thing in common: They do not replace the actual work and are things that will be checked anyways (even the code one, as we know there are still many flaws, but it’s usually great at explaining the code now - not so at writing it). It’s just a convenient method to check your own work - and LLM hallucinations will usually be caught anyway.

      source
• sturmblast@lemmy.world ⁨11⁩ ⁨hours⁩ ago

  I mean… it’s software. It’s only as good as you leverage it.

  source
• Schlemmy@lemmy.ml ⁨1⁩ ⁨day⁩ ago

  Because they don’t know how to use it.

  I work for the government and we’re trialing Copilot too.

  Yesterday I gave copilot several legal documents and our departments long term goals and asked to analyse those documents and find opportunities, legal complications and a matrix of proposed actions.

  In less than 5 minutes I have a great overview to start talks with local politicians. This would have taken me at least a day before AI.

  source

  • MonkderVierte@lemmy.zip ⁨23⁩ ⁨hours⁩ ago

    And you checked that there are no invented points?

    source

    • Schlemmy@lemmy.ml ⁨17⁩ ⁨hours⁩ ago

      Yes. If you feed the documents this goes well enough. It just speeds up the process where I start indexing and gathering information a lot.

      source
    • vala@lemmy.dbzer0.com ⁨14⁩ ⁨hours⁩ ago

      Of course they didn’t

      source
    • vacuumflower@lemmy.sdf.org ⁨21⁩ ⁨hours⁩ ago

      In that case they would have spent “less than 5 minutes” more time than without this, or, one can say, “at least a day” plus “less than 5 minutes”.

      source
  • Kit@lemmy.blahaj.zone ⁨21⁩ ⁨hours⁩ ago

    100%. I’m also trialing Copilot at a medium-sized corpo job and it saves me roughly 12-20 hours of work per week.

    I use it often in PowerShell scripting. It occasionally hallucinates and makes up commands, so sometimes it takes a bit of back and forth to get it to do what I want, but it’s still a hundred times easier than writing from scratch or tweaking+combining similar scripts I find online.

    Probably my favorite part is being able to ask it “Where did I leave off with John on x issue last week?” And it will remind me that I’m supposed to do x and John is supposed to do y. Or even, “I helped a user with this specific issue six months ago. How did I fix it?” and it pulls the exact email and Teams chats outlining what we did, and I can click the link to open those messages and ensure it didn’t misinterperate. Way easier than digging by hand.

    Finally, I absolutely hate making PowerPoints so I’ve been having it make all of my rough drafts from transcription notes in meetings. Super nice time saver.

    Something I’m concerned about and playing with this week is pronoun usage in transcripts. I’m working with our LGBTQ ERG to ensure that we can make Copilot use preferred pronouns for everyone. If it can’t, we’ll need to pull back certain features.

    It’s far from perfect but it genuinely makes my job a lot easier and I’d hate to lose it. I think it will only get better from here.

    source
• Schlemmy@lemmy.ml ⁨1⁩ ⁨day⁩ ago

  I’ve show my coworkers some practical implementations of copilot and that was enough to kickstart the use.

  If you’re composing the same mails a lot, for example, you can ask copilot to make a template text and then when you have to compose the same email again you ask copilot to compose and personalize the mail for you. That’s an awesome funtion.

  I’ve made an agent that answers HR related questions of my team. This saves me and HR a lot of time and they are assured their questions are handled discrete.

  source

  • MonkderVierte@lemmy.zip ⁨23⁩ ⁨hours⁩ ago

    If you’re composing the same mails a lot, for example, you can ask copilot to make a template text and then when you have to compose the same email again you ask copilot to compose and personalize the mail for you. That’s an awesome funtion.

    Uhm, email templates are far older than LLM.

    source

    • vacuumflower@lemmy.sdf.org ⁨21⁩ ⁨hours⁩ ago

      They also don’t make random artifacts out of nowhere

      source
    • Schlemmy@lemmy.ml ⁨19⁩ ⁨hours⁩ ago

      This template adds or deletes links to relevant webpages and adds recent figures when needed.

      We’ve been using templates for years but this adds personality and customisation

      source

      • -> View More Comments
  • kikutwo@lemmy.world ⁨23⁩ ⁨hours⁩ ago

    So there’s this thing called drafts.

    source
• Prior_Industry@lemmy.world ⁨22⁩ ⁨hours⁩ ago

  Seeing a big uptake in use in the education sector. Teachers paying for their own ChatGPT pro license to lesson plan etc.

  Can’t comment at this point if that’s right or wrong, you hope the teachers using it would identify hallucinations etc. But you can see there is already a change occurring.

  source