LOL I should have scrolled down first You said what I said, with fewer words, first.
Comment on Reddit will block the Internet Archive
captainastronaut@seattlelunarsociety.org 3 days ago
As long as the previous collections of archives are still intact. We probably don’t need all of their new spam posts in the wayback machine anyway
Sxan@piefed.zip 3 days ago
hamFoilHat@lemmy.world 3 days ago
It is my understanding that if you block the wayback machine from indexing your site it will also delist the history as well.
Jason2357@lemmy.ca 3 days ago
They do archive sites against the owners wishes when they consider it an important site for public archiving, like some news sites. They are in no obligation to delete the archives and hope they don’t.
tal@lemmy.today 3 days ago
Parties have archived the data from pushshift, which cover a lot of Reddit history.
kagis
academictorrents.com/…/56aa49f9653ba545f48df2e336…
I mean, that won’t have the past 18 months or some low-traffic subreddits, but…
Natanael@infosec.pub 3 days ago
The ability to block crawling is separate from the ability to delist old pages. The latter usually happens after domains change owners