I’ve recently been looking into downloading offline copies of important data, since I don’t expect that today’s freely available information will continue to be freely available and accessible in perpetuity.

One problem I quickly ran into was that e.g. wikipedia downloads are not in an easily browsable format.

I found a project called Kiwix that packages datasets from a variety of free sources, like Wikipedia and Project Gutenberg, along with a reader application that can read these “zim” archives. The different data sources are available via torrents or direct downloads. wiki.kiwix.org/wiki/Content

I’m particularly interested in freely downloadable archives of scientific papers. A lot seems to be paywalled, or at least free-account-walled, even though the papers themselves are theoretically open-access. I would love to know of any sources out there to download an entire database locally.