From skimming that linked page, I think that this download perhaps doesn’t include recent pages? Because in the section talking about enterprise stuff, it mentions the paid API for recent articles
Comment on Wikipedia urges AI companies to use its paid API, and stop scraping
usernameusername@sh.itjust.works 1 week ago
I don’t get it though… Why would any company use this when Wikimedia also offers a download of the entirety of Wikipedia, for free?
Maybe it’s just that if the AI companies don’t know, they can hopefully get a little money from them?
AnarchistArtificer@slrpnk.net 1 week ago
usernameusername@sh.itjust.works 1 week ago
It seems you’re right, I’m just dumb and didn’t read ths article I linked
Crashumbc@lemmy.world 1 week ago
You think AI companies care what they scrape. Their system is set up to scrape anything it can get.
usernameusername@sh.itjust.works 1 week ago
Oh I know, I was just thinking that if the AI companies will make an exception for Wikipedia (by paying) like the Wikimedia people think, they could also download the complete thing for free. But yeah they probably won’t do any of that so this was kinda useless I think
BanMe@lemmy.world 1 week ago
They can scrape an ongoing log of interactions between editors about the articles themselves, which is probably fairly worthwhile content honestly. More content there than in articles probably as well.