Comment on Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi...

• killerinstinct101@lemmy.world ⁨1⁩ ⁨year⁩ ago

  This is what was addressed at the start of the comment, you can just roll back to a previous version. It’s heavily ingrained in CS to keep every single version of your software forever.

  source

  • CaptainAniki@lemmy.flight-crew.org ⁨1⁩ ⁨year⁩ ago

    I don’t think it’s that easy. These are vLLMs that feed back on themselves to produce “better” results. These models don’t have single point release cycles. It’s a constantly evolving blob of memory and storage orchestrated across a vast number of disk arrays and cabinets of hardware.

    source

    • Lazylazycat@lemmy.world ⁨1⁩ ⁨year⁩ ago

      Exactly this, that’s why Loab exists forever now.

      source
    • agent_flounder@lemmy.one ⁨1⁩ ⁨year⁩ ago

      Even so, surely they can take snapshots. If they’re that clueless about rudimentary practices of IT operations then it is just a matter of time before an outage wipes everything. I find it hard to believe nobody considered a way to do backups, rollbacks, or any of that.

      source