Comment on Self hosting Sunday! What's up and how long?
dotslashme@infosec.pub 1 week ago
I had a weird issue with a server SSD disk.
6 months ahead of scheduled swap, it didn’t die, it just started reading and writing really sluggishly, making the whole server behaving really weird. Disk smart statistics looked healthy and disk self tests passed with flying colors. Anyway, had to swap it early and do a re-install of the OS.
The rest of my cluster temporarily took over running some pods and only saw downtime for a few pods that were dependent on some disks in the failing server.
I guess the incident has restarted my interest in distributed storage.
tofu@lemmy.nocturnal.garden 1 week ago
Blergh, how did you pinpoint it?
dotslashme@infosec.pub 1 week ago
More luck than anything really. It was probably because it had 6 months left and the fact that reading and writing felt slow. Everything else behaved normally and buying a new disk was an educated guess that turned out to be the correct choice.