[deleted]
proper archiving is surprisingly difficult, esp the time capsule idea. maybe this gets you started
[deleted]
Submitted 3 months ago by DeathByBigSad@sh.itjust.works to nostupidquestions@lemmy.world
Comments
drre@feddit.org 3 months ago
Creat@discuss.tchncs.de 3 months ago
Hard drives that aren’t used will get data errors over time. Usually for data storage this is counteracted with what’s called a “scrub” every so often (like few months). This just means the whole drive content is read, and the drive itself will figure out if any areas have a “weak signal”, and just rewrite that part.
Having only 1 drive without any mirror and without any way to detect potential errors (let alone a way to correct them) is a recipe for disaster.
Blackmist@feddit.uk 3 months ago
And definitely don’t use an SSD.
traceur201@piefed.social 3 months ago
I think if you want 10+ years with high assurance you probably want to burn the data to archival quality BD-R disks (not the dye based ones)
The right spinning platter hard drives might have a decent chance to make it 10 years but there's a lot of possible failure modes and also a decent chance that when you try spinning it back up it gives nothing but read errors.
For cases for "only" 10-30 years I might pick a pelican-like case inside a makeshift wooden coffin-like outer layer. For longer I'd probably use a metal box like an ammo box inside the plastic case and a stone outer layer instead of wood
Thorry84@feddit.nl 3 months ago
You would need to investigate the soil you put the wood in, in order to select the correct wood and wood treatment. The wrong kind of wood in certain soils can be broken down in weeks to months. Getting wood to last years is tricky and depending on the soil could even be impossible.
Cethin@lemmy.zip 3 months ago
An ammo box is probably cheaper than a pelican case. I’d go for that no matter what.
LordWiggle@lemmy.world 3 months ago
You must have an amazing porn collection. You can store it on my NAS
floo@retrolemmy.com 3 months ago
How much better medium would be tape back up, or possibly Blu-ray discs. Either one would last a lot longer than a hard drive.
dysprosium@lemmy.dbzer0.com 3 months ago
A so-called M-disk is rated for 1000 years. Artificial lab tests could at least ‘confirm’ a few hundred years. Amazing shit.
Add encryption to it, keep the keys safe (perhaps on another M-disk) and you’re set!
floo@retrolemmy.com 3 months ago
Those are used by US government agencies, such as the department of defense for archival purposes. They are rated for 100+ years.
cam_i_am@lemmy.world 3 months ago
If this is a real problem you have, and not just a thought experiment, I think rather than burying the data on some unreliable medium, your best bet is to just pay someone to store it for you offshore, away from the dictatorship you mentioned.
There are plenty of consumer-grade cloud storage services. I’m sure there are more niche ones specifically for long-term archival as well, which would usually be cheaper per bit, per-year, if you don’t need to access the data regularly.
Jimmycrackcrack@lemmy.ml 3 months ago
Would you have to worry about the records of your ongoing business with these data storage providers though?
cam_i_am@lemmy.world 3 months ago
Yes that’s true. There may also be issues just with getting money out of the country to make the required payments to the storage providers. Either due to local restrictions or international sanctions.
perviouslyiner@lemmy.world 3 months ago
For a somewhat recent real-world example of hiding things in this kind of situation, maybe look at how ‘paramilitary’ people in Northern Ireland hid things by putting them in walls and then decorating the wall.
Maybe some “outlet” in your house is actually the connector to the NAS sealed into a void space?
dual_sport_dork@lemmy.world 3 months ago
I’m going to buck the trend here and suggest a really physical storage medium: Print your data out. Or laser engrave it onto sheets of metal or polymer, or whatever you want to do. If you just print pokey old black and white ones and zeros as square pixels on a sheet of 8.5x11" paper at a humble 72 DPI you can store a shade under 47 kilobytes per page without having to resort to any additional trickery. Maybe a kB or two less if you need to leave margins. How much data are you really trying to store?
In a sealed container in the dark you could easily make paper last hundreds of years (we have perfectly intact books sitting on ordinary shelves from the 1800s already), and if you wanted to print on Tyvek or something it’d probably endure thousands.
Reading this back would not be a plug-and-play solution but would have the added advantage of being a purely optical process rather than having to interface with antique storage device electronics on whatever computer you may be using 30 years from now. All you’d need is sheet feed scanner or in a pinch any sufficiently high resolution camera, and the ability to run some kind of programming environment to run a script to read those pixels back into file data.
Maybe this wouldn’t be great for archiving your collection of 4k ultra-definition porn, but it’d be absolutely sufficient for storing text and executable data for small programs, plans and schematics, other knowledgy sciency data, and even images… with the added benefit of, if any gestapo thug happens to find this early and dig it up he won’t be able to ascertain what that image is just by looking at the piece of paper.
Atlas_@lemmy.world 3 months ago
If you actually want to use paper… QR codes. The format is simple, broadly distributed, and has error correction built in. It’ll make the whole process a lot easier than trying to roll something yourself.
dual_sport_dork@lemmy.world 3 months ago
Another poster here suggested the High Capacity Color Barcode as well, which ought to already have some implementations available somewhere and sports an even higher data density if you’re willing (or able) to deal with color.
QR codes are limited to being square in aspect ratio (other than the not terribly helpful “micro rectangular QR” format) and have a maximum payload of ~3kB each. This may not be a great fit for plain consumer paper with a rectangular aspect, and you’d need to jigger some manner of batch reader so’s you don’t drive yourself insane recovering the data. Neither is an insurmountable problem; I’m just thinking out loud, here.
RoyaltyInTraining@lemmy.world 3 months ago
I once heard that some printers print (almost) invisible yellow dots on pages, containing data which helps authority track down whoever printed the page. That might be a risk if the data is really sensitive.
jasoman@lemmy.world 3 months ago
This is true.
DirigibleProtein@aussie.zone 3 months ago
So much data has already been lost due to bitrot caused by magnetic loss and plastic breakdown. Most consumer grade storage will break down and start to lose data within a decade. Even if the data survives, will the operating system and software be available in the future to read the media? Surprisingly, the best way to preserve data long term is to print it on paper. Or write it to a gold record and send it into space.
DeathByBigSad@sh.itjust.works 3 months ago
The english wikipedia is only 100GB, so its easy to fit on digital storage, but printing it on paper is gonna take a whole building of physical space to even fit it.
Jimmycrackcrack@lemmy.ml 3 months ago
traches@sh.itjust.works 3 months ago
I’d go with optical media here. Probably multiple capsules.
- M-Disk (DVD if it will fit, otherwise Blu-ray)
- Make an encrypted archive of your data. Strong password - I suggest diceware with 8 or more words so you might remember it in 30 years
- Use DVDisaster to add parity data. You sacrifice some space, but you get error tolerance in exchange
- Wrap the disks up in good jewel cases, well sealed plastic, along with some good big silica gel desiccant packs.
- Put all that in the smallest durable, airtight container you can
- stash somewhere it probably won’t be disturbed for a few decades. Memorize.
- destroy all evidence you did this.
some_guy@lemmy.sdf.org 3 months ago
No way. Optical media suffer bitrot at a high rate compared to magnetic media. And the means to read it are quickly going obsolete.
Twinklebreeze@lemmy.world 3 months ago
That’s what the m-disk is for I assume.
traches@sh.itjust.works 3 months ago
It’s pretty dependent on humidity and temperature, so a DVD buried in a well sealed plastic bag with a desiccant packs is actually in good conditions. No light, generally cool, and low humidity are perfect.
A hard drive has a lot of moving parts that must work and are basically impossible to replace. With optical media you’re just storing the platters, and I’m sure you’ll still be able to track down a drive somewhere. You can still find VHS players and those have been obsolete for 25 years.
ooterness@lemmy.world 3 months ago
This is terrible advice. Writable DVDs degrade quickly, even if they’re stored away from sunlight and heat. Every single one of my burned DVDs from more than s few years back is completely unreadable.
traches@sh.itjust.works 3 months ago
Do you remember what kind they were? For awhile they made them with organic dyes and those died quickly. I believe they stopped producing those, and the inorganic ones are supposed to be much better.
grayautumnday@lemmy.4d2.org 3 months ago
What about tape drives? You can still get them, and I have come across articles a few times (which I can’t find on a quick search, but I only use DDG now) saying that tape drives written 30-45 years ago, carefully conserved, were still readable after all that time.
traches@sh.itjust.works 3 months ago
I looked into tape drives for my own backups and they don’t make sense unless you’re working with double digit terabytes. We’re talking used old enterprise gear with weird form factors and connectors, I never found something like an external USB tape drive for a reasonable price.
perviouslyiner@lemmy.world 3 months ago
Sorry if this is obvious to everyone, but how would having a hidden hard disk help with living in a dictatorship?
Couldn’t you just let someone in another country take care of archiving it?
Lumisal@lemmy.world 3 months ago
It’s 8tb of porn and the government will be banning it, and they’re hoping it’ll pass with time like prohibition did?..
ivanafterall@lemmy.world 3 months ago
If it were me, I’d be skimming a little to sell on the side to take advantage of those black market prices.
Vintage Hulk Fucks Black Widow GIF - $500
sunstoned@lemmus.org 3 months ago
My take – OP is an anti-authoritarian time traveler. Go get em! I hope your data stays safe.
P.s. - want to drop me some winning lottery numbers? My dms are open
InFerNo@lemmy.ml 3 months ago
They key is to diversify. Use different types of storage media, and duplicate your efforts and bury then duplicates somewhere else.
If you can choose only 1 I would choose tape archives. Vacuum seal all your media, whatever they may be. Throw in some of those dehumidifier packets. Moisture will be your biggest enemy.
If possible, also add the means to be able to read your media after a long time. Add a couple of raspberry pi computers, vacuum sealed and dehumidified-by-packets again, and usb readers or HATs for the media you chose (though I doubt you will find a cheap tape drive with USB connection, the only option I found was £9000).
Over the years, as new technology gets developed, in particularly interface connectors that will replace USB, I would add converters if possible or just keep them around. Nothing suspicious about having some USB/sata/sas to <new technology> converter in your house.
Berttheduck@lemmy.ml 3 months ago
You could look at fire safe boxes for document storage. Those are usually pretty solid. You would want to bag up the drive inside an anti static bag and probably put a couple of those little water absorbing silicone packets in there as well. If access isn’t an issue then maybe some sealant around the seams to keep it more water tight.
Magnetic tape would be better for long term storage as well I think. Those have longer storage stability. I don’t know how long an unplugged hard drive will reliably store information.
Animals could dig it up but probably wouldn’t as it wouldn’t smell like food. Depth wise I’d go for at least a couple feet deep, the traditional 6 is a surprisingly deep hole and temperature gets more consistent the deeper you go (at least with readily available tools, it eventually starts to get hot again).
Please note totally random opinion with very little experience with long term data storage. Thanks for the fun thought experiment, I hope things get better and you don’t need your backup data.
some_guy@lemmy.sdf.org 3 months ago
Don’t bury it. And don’t count on ten years. Thirty years guarantees the media won’t be physically compatible with future devices. How would you read a floppy disk from 1995 today? You’d be able to find a USB floppy drive, probably, online. Good luck having the disk be in a format that a modern OS understands. You’d need specialty software for that.
Get two spinning disk drives from major brands like Western Digital or Toshiba (not Seagate, for sure). Get different brands to reduce risk of failure from a manufacturing issue (as in, two from the same batch are likely to have the same failure if there was a production issue).
Send one somewhere abroad where it can be stored in a safe deposit box (hopefully, you know someone who lives in a free-er country). Plan to exchange it with a freshly written drive every three years. Go back and forth like this, completely rewriting the data each time to minimize the chances of bit-rot (look up this term to understand why you’re rewriting and exchanging the drives).
This will also address files formats that evolve and eventually become incompatible with future software (thinking proprietary things, not plain text, jpegs, or standardized media files). I did something similar having a family member store music that I recorded (my own, not ripped CDs) in a different state in case of natural disaster at home.
All of this can be done pretty cheap. $200 bucks should cover both drives and at least a couple of years of physical storage at a bank. International shipping will probably be the biggest cost, especially over time.
BeardedBlaze@lemmy.world 3 months ago
All modern OSs can read fat16 or fat32, not sure what you think floppy disks used.
db2@lemmy.world 3 months ago
Neither of those, my guy.
some_guy@lemmy.sdf.org 3 months ago
It would have been more accurate for me to frame it as file formats only.
rbos@lemmy.ca 3 months ago
Not difficult, or even expensive, to find a working 20 year old machine with a 3.5" FDD. Also I work at a library and we keep a couple of well bagged USB floppy drives around for profs who occasionally need data retrieval. Hasn’t happened in a couple years though. We also have an old Dell for 5.25".
Randomgal@lemmy.ca 3 months ago
Or just let it go. Enjoy the present and realize you can’t predict the future.
Any situation when an arrangement like this becomes useful, means you’ll have much worse and much more important things to concern yourself with.
normonator@lemmy.ml 3 months ago
Holy shit toshiba hard drives are fucking awful, and floppies are still not hard to read today.
I swear it’s half the reason people are mad at Synology. There is no way to buy a “Synology” drive without the chance of getting a Toshiba drive, just return and reorder until you get decent drives.
hardcoreufo@lemmy.world 3 months ago
Floppies read just fine today with usb drives. I read and write floppies for a bunch of weird, old computers at work just fine. I expect that sata hardware will still be pretty accessible in 30 years even if it isn’t standard on future PCs.
Swedneck@discuss.tchncs.de 3 months ago
i think your best bet would just be to brute force it: get a bunch of different media (usb-drives, CDs, hard drives, whatever you can get your hands on and ideally from different brands) and just put the same data on all of them, then wrap in a series of plastic bags that you try to put a vacuum on, put in the most durable water and ideally airtight container you can get your hands on, then again wrap that in some plastic bags because why not.
Then bury all that as deep as you can and surround it in rocks, especially i think you’ll want a bunch of rocks on top of it. And for extra points repeat all this as many times as possible in different locations.All this is just to compound the chances that at least one of the copies of the data will survive, and even if that fails you’ll hopefully end up with enough data being intact across the different storage mediums that you can piece the data together somehow, i’m not sure how precisely you’d do that but it’s at least possible to figure out so long as the data is there.
Typotyper@sh.itjust.works 3 months ago
There definitely is an advantage to different media types. Each technology has their own limitations. Tape back up, SSD, USB drive, DVD and HD with spinning platers
Blackmist@feddit.uk 3 months ago
SSD would be 100% dead unless you buried it with a power source.
rbos@lemmy.ca 3 months ago
A raid6 array across a collection of separate disks might do it.
Pencilnoob@lemmy.world 3 months ago
If it’s small you might try printing the files on archival paper with archival ink. Then you can put copies in multiple safe deposit boxes. Also you could bury copies rolled up in plastic water bottles. I think those are unlikely to degrade anytime soon. Or glass bottles with plastic lids.
sefra1@lemmy.zip 3 months ago
The issue with hard drives is that they tend to fail even on ideal conditions and even when powered down. Yes I’ve lost very important data to a powered down hard drive.
While it’s possible to recover information on a hard drive as long as the plates themselves aren’t damaged, that requires very expensive specialised tools and skills. Which probably wouldn’t be available in a scenario where the information on the drive would be of any value.
DVD-R (and probably consequentially Blu-Rays) aren’t any better in my experience, I’ve lost more data to DVD-R than to hard drives actually. Even when stored in low light conditions they tend to just stop reading.
However optical media has one big advantage here, is that the discs themselves are cheap, so instead of having all your digital eggs in the same basket, you spread them over several discs and while some information may be lost, others may survive.
Now, here’s an interesting thought, with digital data, the data either reads or doesn’t read, the so called digital cliff, may become partially corrupted and other parts still read, but after the corruption gets past a certain threshold all information is lost.
With analogue equipment even after severe signal degradation the contents while very deteriorated may still be perceptible, forwardermore an analogue signal is much easier to decode in the event that you need to restart
civilisationbuilding tech from scratch and don’t have access to the very very specific specifications of something like the audio codec or the filesystem.You can probably hack a rudimentary cassette player together from very simple components, all you need is a tape head (a coil), a motor (a coil and a magnet), and an amplifier (a transistor or vaccum tube). (I’m probably oversimplifying here).
Overall I think the most important thing is having redundancy, or if redundancy isn’t possible at least don’t have all eggs in the same basket, instead of having everything in a single 8TB HDD, to try spread them into smaller 512GB ones, or DVDs or flash drives or all of the above. And don’t store them all in the same location, if an area gets flooded or someone builds a building on top, you’re only losing a small part of the information.
discosnails@lemmy.wtf 3 months ago
Ammo can with silica gel beads.
Duamerthrax@lemmy.world 3 months ago
Heavily waxed and buried in a dry place, preferably somewhere where water doesn’t flow or collect.
phoenixz@lemmy.ca 3 months ago
Storage media won’t survive that long. Hard drive, when used, last about 5 years, give or take. Unused, I have no idea how long the data will stay consistent but I would not count on anything beyond 10 years
FuglyDuck@lemmy.world 3 months ago
so, I would suggest talking with an archivist. Many libraries will have archivists on the payroll (Or know one, anyways) and they’d likely be happy to talk about archival methods.
personally, what I would do- and I make no guarantees that it will work for a decade- is to seal the hard drive (or whatever media,) inside a vacuum bag with a shitload of silica desiccant gel. maybe double bag it with even more silica gel, then place it inside a pelican case. if you double bag, splurge on the indicator stuff and let it sit for a week.
but I’m not an archivist, and they may laugh at my suggestion.
Cethin@lemmy.zip 3 months ago
I have a question. Is this for you in the future, or for someone who may find it? If it’s the latter, and it’s just information you want to store, not media, I’d just go with paper. Storing digital data is both hard and error prone, and it also requires them to have the technology and power to read it. If things really go to hell, this isn’t a guarantee. Paper ensures they can at least view it no matter what. It’ll degrade eventually, but it’ll hold up better than digital.
zorro@lemmy.world 3 months ago
Any thoughts on tape? Lto tape is designed for 10+ years
SkunkWorkz@lemmy.world 3 months ago
Yeah in a controlled environment. Doubt it can last the promised lifespan when it’s buried underground.
altkey@lemmy.dbzer0.com 3 months ago
Drop a thumb stick (mechanical failure) into a plastic zip-lock in a vacuum (oxygen) then into a metal thermos mug with water (pressure and radiation) then dig it really deep (accidential discovery and weather). By the time it deteriorates you’d have problems finding USB interfaces to plug it in. The location itself is largely irrelevant, but I’d recommend some place far from human-occupied places.
The authoritarian state problem isn’t solveable, but you can defend it by obscurity, like not leaving a trace of thinking about this info cache, or leaving too many of these caches to reliably dig up all of them.
Swedneck@discuss.tchncs.de 3 months ago
flash storage does degrade though, sure it’s presumably slowed down by a stable environment without oxygen, but i can’t imagine it lasting more than 50 years
altkey@lemmy.dbzer0.com 3 months ago
I find it more reliable as it has less mechanical parts, but I am curious if someone did a scientifical aproximation of how long it would last.
NostraDavid@programming.dev 3 months ago
If it’s a Type-A, add a little Type-A to Type-C converter in the bag as well. Just in case.
NoodlePoint@lemmy.world 3 months ago
Most of those ideas are not feasible with a very low budget you want because eventually rot will get to the hard drive and thus making the contents unreadable. So – depending on what you want to preserve – it’s either writable media or printed out in acid-free paper or in microdot negative film, and of those methods, only print media – written, typed, from a copier, or with a laser printer – might as well be cheap.
JackbyDev@programming.dev 3 months ago
You don’t use a hard drive. USB sticks would be easier and more likely to survive I think. SD cards are another option.
mic_check_one_two@lemmy.dbzer0.com 3 months ago
Flash isn’t a good long-term storage option. It relies on an electrical charge to store the data, and will discharge over time. It’s literally physically storing electrons, but those electrons are constantly trying to escape. Good flash may last 7-10 years without being plugged in, but the standard off-the-shelf stuff will be dead much quicker.
Tape or M-DISC are the gold standards, though both are more expensive than flash. Tape is by far the single most resilient method; it can even be reconstructed if it is physically damaged. But it’s also the least convenient and most expensive. M-DISC is a nice middle ground. It’s essentially just a burned disc, but made with materials that won’t rot over time like standard burned discs will. So storage is as simple as storing regular discs. Though if you’re truly trying to apocalypse-proof it, you’d probably want to consider bunkering/burying them somewhere to protect from physical destruction.
SCmSTR@lemmy.blahaj.zone 3 months ago
Wait… Tape is more resilient than m-disc? What? How? Doesn’t tape rot or melt or decay faster than m-disc?
DeathByBigSad@sh.itjust.works 3 months ago
But those rely on electricity to keep their bits in the right position, so leaving them unplugged for a long time means they could lose the charge on the bits.
JackbyDev@programming.dev 3 months ago
Uhhh… I don’t know about all that. But putting something with moving parts like a hard drive into dirt unless you really really seal it well is not a good bet.
leftzero@lemmy.dbzer0.com 3 months ago
Good flash memory might last a decade, maybe a bit more.
Average flash memory probably won’t.
Toes@ani.social 3 months ago
Probably want to encode it on a WORM tape. (Suggestion used LTO drives on eBay)
Then store it in the centre of a sealed medium iron box filled with silica. (Take care not to damage the tape, without trapping moisture.)
I’d imagine it would work well if you can keep the hardware to use it functional.
chocrates@piefed.world 3 months ago
Hell, encode it into stone tablets. Those would last forever, but read time would be awful.
DeathByBigSad@sh.itjust.works 3 months ago
“Bring me the tablet”
“No, not the ipad, the stone tablet that I buried underground”
CatDogL0ver@lemmy.world 3 months ago
As someone who has lost hard drive in the past, encrypt and back up to the cloud.
It is the safest way.
Atlas_@lemmy.world 3 months ago
Does it need to be physical? I’d expect data on a well funded S3 account or a tar snap account to live 30 years
HereIAm@lemmy.world 3 months ago
I wouldn’t trust cloud storage in case of a dictatorship.
thelittleblackbird@lemmy.dbzer0.com 3 months ago
You can not.
There is not a safe and reliable way to store digital information for such big time span while off.
The maximum you could get is some programmable eeprom and usually no vendor will bet that the information is accesible after 15 years while power off.
But once this is said, there a re few things you con do to maximize your chances.
From the technology point of view everything that is using old nand-flash technology should give you decent chances after 15 years power off. To ensure better probabilities use a fs with possibilities of storing recovery /parity/ checksum data. And try to store in a environment with minimum changes in temp, humidity and radiation (electromagnetic, solar).
And cross some fingers
Deestan@lemmy.world 3 months ago
Hard drives aren’t rated for 30 years, though. Even in optimal conditions, they’d deteriorate.
masterofn001@lemmy.ca 3 months ago
So, an atomic powered RAID array with SMART corruption correcting code attribute in a timed replacement sequence of a series of single platter, low RPM, drives?
But apparently, using a simple archival quality DVD+R or Blu-ray.
Apparently verbatim gold archive DVD+r has been rated for between 32 and 127 years with a minimum 18.
Some Blu-ray from a few corps is rated at 50 years.
Under ideal conditions.
However, I’ll stick to my crystal skulls and their magic alien data storage.
DeathByBigSad@sh.itjust.works 3 months ago
Brb, gonna steal the plutonium from the Libyans.
Jimmycrackcrack@lemmy.ml 3 months ago
Ok but now how do we keep the Bluray drive and any additional materials to make it itself compatible with future hardware it’ll have to interact with, in working order for the same timespan as the media it reads?
Deestan@lemmy.world 3 months ago
Archive level Blue-Rays sound interesting!
But note that any drive based solution with RAID or anything runs into the problem that the drives all age at the same time. Once one drive fails, the others are close to failing also.