Comment on VCs are starting to partner with private equity to buy up call centers, accounting firms and other "mature companies" to replace their operations with AI

cyrano@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

• Initiateofthevoid@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

  The idea of AI accounting is so fucking funny to me. The problem is right in the name. They account for stuff. Accountants account for where stuff came from and where stuff went.

  Machine learning algorithms are black boxes that can’t show their work. They can absolutely do things like detect fraud and waste by detecting abnormalities in the data, but they absolutely can’t do things like prove an absence of fraud and waste.

  source

  • vivendi@programming.dev ⁨1⁩ ⁨week⁩ ago

    For usage like that you’d wire an LLM into a tool use workflow with whatever accounting software you have. The LLM would make queries to the rigid, non-hallucinating accounting system.

    I still don’t think it would be anywhere close to a good idea because you’d need a lot of safeguards and also fuck your accounting and you’ll have some unpleasant meetings with the local equivalent of the IRS.

    source

    • pinball_wizard@lemmy.zip ⁨1⁩ ⁨week⁩ ago

      The LLM would make queries to the rigid, non-hallucinating accounting system.

      And then sometimes adds a halucination before returning an answer - particularly when it encournters anything it wasn’t trained on, like important moments when business leaders should be taking a closer look.

      There’s not enough popcorn in the world for the shitshow that is coming.

      source

      • vivendi@programming.dev ⁨1⁩ ⁨week⁩ ago

        You’re misunderstanding tool use, the LLM only queries something to be done then the actual system returns the result. You can also summarize the result or something but hallucinations in that workload are remarkably low (however without tuning they can drop important information from the response)

        The place where it can hallucinate is generating steps for your natural language query, or the entry stage. That’s why you need to safeguard like your ass depends on it. (Which it does, if your boss is stupid enough)

        source

        • -> View More Comments
    • futatorius@lemm.ee ⁨1⁩ ⁨week⁩ ago

      The LLM would make queries to the rigid, non-hallucinating accounting system.

      ERP systems already do that, just not using AI.

      source

      • vivendi@programming.dev ⁨1⁩ ⁨week⁩ ago

        But ERP is not a cool buzzword, hence it can fuck off we’re in 2025

        source
  • futatorius@lemm.ee ⁨1⁩ ⁨week⁩ ago

    LLMs often use bizarre “reasoning” to come up with their responses. And if asked to explain those responses, they then use equally bizarre “reasoning.” That’s because the explanation is just another post-hoc response.

    Unless explainability is built in, it is impossible to validate an LLM.

    source
• Korhaka@sopuli.xyz ⁨1⁩ ⁨week⁩ ago

  How easy will it be to fool the AI into getting the company in legal trouble? Oh well.

  source

  • pinball_wizard@lemmy.zip ⁨1⁩ ⁨week⁩ ago

    Some would call it effortless, even.

    source
  • Meron35@lemmy.world ⁨1⁩ ⁨week⁩ ago

    NYC’s AI chatbot was caught telling businesses to break the law. The city isn’t taking it down | AP News - apnews.com/…/new-york-city-chatbot-misinformation…

    source
• Melvin_Ferd@lemmy.world ⁨1⁩ ⁨week⁩ ago

  Hey boss. Think they’re using chatgpt for that?

  source
• vivendi@programming.dev ⁨1⁩ ⁨week⁩ ago

  This is because auto regressive LLMs work on high level “Tokens”. There are LLM experiments which can access byte information, to correctly answer such questions.

  Also, they don’t want to support you omegalul do you really think call centers are hired to give a fuck about you? this is intentional

  source

  • Repelle@lemmy.world ⁨1⁩ ⁨week⁩ ago

    I don’t think that’s the full explanation though, because there are examples of models that will correctly spell out the word first (ie, it knows the component letter tokens) and still miscount the letters after doing so.

    source

    • vivendi@programming.dev ⁨1⁩ ⁨week⁩ ago

      No, this literally is the explanation. The model understands the concept of “Strawberry”, It can output from the model (and that itself is very complicated) in English as Strawberry, jn Persian as توت فرنگی and so on.

      But the model does not understand how many Rs exist in Strawberry or how many ت exist in توت فرنگی

      source

      • Repelle@lemmy.world ⁨1⁩ ⁨week⁩ ago

        I’m talking about models printing out the component letters first not just printing out the full word. As in “S - T - R - A - W - B - E - R - R - Y” then getting the answer wrong. You’re absolutely right that it reads in words at a time encoded to vectors, but if it’s holding a relationship from that coding to the component spelling, which it seems it must be given it is outputting the letters individually, then something else is wrong. I’m not saying all models fail this way, and I’m sure many fail in exactly the way you describe, but I have seen this failure mode and in that case an alternate explanation would be necessary.

        source

        • -> View More Comments