I made a Hacker News post about human knowledge databases and got some interesting data I’d like to share back to this thread.
- Wikipedia Single 100GB File Thanks to @hersh@literature.cafe
- sci-hub.wf
- Arxiv Download Script
- wholeearth.info
- the-eye.eu
Comment on FreeBSD, GhostBSD, OpenBSD, Dragonfly BSD, Firefox Hardening - FOSS - git clone
elias_griffin@lemmy.world 11 months ago
Yes, I’m serious about my mission statement in the beginning and I have some more ideas. First there is a Linux OS that installs all kinds of Educational Software, like Encyclopedia, Maps, Learning Tools that is all available offline in the full 17GB Full Version. It’s called Endless OS (no affilitions) and here is the excerpt.
Multi-language system, pre-loaded with apps in English including games, productivity software, reference materials like Cooking, Farming, Health, Travel, and educational materials like a robust Encyclopedia.
It would be great if all of us could have some of the civilization important databases on this BSD installation of yours. Please contribute a downloadable database file or file set you know of. I’ll start.
Downloadable Wikipedia Database Encyclopedia Britannica All Volumes
elias_griffin@lemmy.world 11 months ago
hersh@literature.cafe 11 months ago
I’ve recently been looking into downloading offline copies of important data, since I don’t expect that today’s freely available information will continue to be freely available and accessible in perpetuity.
One problem I quickly ran into was that e.g. wikipedia downloads are not in an easily browsable format.
I found a project called Kiwix that packages datasets from a variety of free sources, like Wikipedia and Project Gutenberg, along with a reader application that can read these “zim” archives. The different data sources are available via torrents or direct downloads. wiki.kiwix.org/wiki/Content
I’m particularly interested in freely downloadable archives of scientific papers. A lot seems to be paywalled, or at least free-account-walled, even though the papers themselves are theoretically open-access. I would love to know of any sources out there to download an entire database locally.
elias_griffin@lemmy.world 11 months ago
I was thinking the exact same thing, thanks for the awareness! Tangentially, problem with Wiki is it’s excellent for Who, What, Where, but is nearly totally bereft of how to do anything.
0ddysseus@lemmy.world 11 months ago
Aha! I went down this road recently.
I have the main full Wikipedia plus extras on my file server accessible on the home land through kiwix.
It was really simple and painless to set up. It also includes wikihow which is the How you’re after
These things aren’t the greatest sources for anything of course but they cover pretty much everything you might want to know in a general “archiving human knowledge at home” sense.
Very happy with it all and have bookmarks on all the devices at home now.
10/10