What software does Internet Archive run?

Comments

• kittykittycatboys@lemmy.blahaj.zone ⁨9⁩ ⁨months⁩ ago

  afaik, archive.org isnt open source. id recommend something like archivebox.io

  source

  • possiblylinux127@lemmy.zip ⁨9⁩ ⁨months⁩ ago

    Archive box is a piece of software and the Internet archive is a organization that is focused on predicting the content on the internet.

    The Internet Archive has PBs worth of data. I doubt any home user could manage that.

    source

    • z00s@lemmy.world ⁨9⁩ ⁨months⁩ ago

      archive

      predicting

      source

      • -> View More Comments
    • kittykittycatboys@lemmy.blahaj.zone ⁨9⁩ ⁨months⁩ ago

      i dont think op is looking to mirror archive.org, my take was that they wanted someyhing like archive.org but selfhosted and for personal / small-scale use

      source

      • -> View More Comments
  • avidamoeba@lemmy.ca ⁨9⁩ ⁨months⁩ ago

    Oh yes, this looks like a winner. Thanks!

    On a far side of the moon note, I wonder if ActivityPub could be used to federate multiple archiveboxes to create a more resilient Internet Archive. 🤔

    source

    • density@kbin.social ⁨9⁩ ⁨months⁩ ago

      a network between networks to make them more resilient i think you've just invented the arpanet?.

      source
  • Dehydrated@lemmy.world ⁨9⁩ ⁨months⁩ ago

    +1 for ArchiveBox

    source
• atzanteol@sh.itjust.works ⁨9⁩ ⁨months⁩ ago

  I believe they used heritrix at one point. The important bit is that there is a special archive format that they use which is a standard. There are several tools that support it - it allows for capturing a website in a ‘working’ condition with history or something. I’m a bit fuzzy on it since it’s been some time since I looked into it.

  source

  • avidamoeba@lemmy.ca ⁨9⁩ ⁨months⁩ ago

    It seems like all of their software is in the parent account of heritrix - github.com/orgs/internetarchive/repositories?type….

    source
• laserjet@lemmy.dbzer0.com ⁨9⁩ ⁨months⁩ ago

  github.com/ArchiveBox/…/Web-Archiving-Community has a wide range of options

  source
• ericjmorey@programming.dev ⁨9⁩ ⁨months⁩ ago

  Does Linkwarden fit your intended use?

  source

  • avidamoeba@lemmy.ca ⁨9⁩ ⁨months⁩ ago

    Kind of. Linkwarden seems to save as PDF. That’s better than nothing, however preserving a functional copy of the pages would be better. Archivebox seems to do this.

    source
• possiblylinux127@lemmy.zip ⁨9⁩ ⁨months⁩ ago

  I don’t know for certain but I’m sure they run lots of different software. They have PBs of data.

  source