Something interesting that I found: according to dolphin, many folders have many GB extra (e.g. 52GB vs 66GB for documents folder which is kind of crazy) while filelight records 52GB vs 112GB for documents folder, which if true, is kind of insane. Using du -sh records 53G vs 136G (they’re the same when using --apparent-size, weird. Specifically for Godot directory, it’s 3.8GB vs 41 GB!!!)!!! Files like videos and games seem to be about the same size, while Godot projects with git are much bigger. Weird.
Comment on When using rsync to backup my /home folder to an external 1TB SSD, I run out of space, how??
bleistift2@sopuli.xyz 12 hours ago
Personally, I have no more tips that those that have already been presented in this comment section. What I would do now to find out what’s going on is the age-old divide-and-conquer debugging technique:
Using rsync or a file manager (yours is Dolphin), only copy a few top-level directories at a time to your external drive. Note the directories you are about to move before each transfer. After each transfer check if the sizes of the directories on your internal drive (roughly) match those on your external drive (They will probably differ a little bit). You can also use your file manager for that.
If all went fine for the first batch, proceed to the next until you find one where the sizes differ significantly. Then delete that offending batch from the external drive. Divide the offending batch into smaller batches (select fewer directories if you tried transferring multiple; or descend into a single directory and copy its subdirectories piecewise like you did before).
In the end you should have a single directory or file which you have identified as problematic. That can then be investigated further.
sbeak@sopuli.xyz 10 hours ago
bleistift2@sopuli.xyz 10 hours ago
These differences really are insane. Maybe someone more knowledgeable can comment on why different tools differ so wildly in the total size they report.
I have never used BTRFS, so I must resort to forwarding googled results like this one.
Could you try
compsize ~? If thePerccolumn is much lower than 100% or theDisk Usagecolumn is much lower than theUncompressedcolumn, then you have some BTRFS-specific file-size reduction on your hands, which your external exFAT naturally can’t replicate.sbeak@sopuli.xyz 10 hours ago
percentage of total is 83% (292G vs uncompressed 349G apparently)
bleistift2@sopuli.xyz 9 hours ago
It’s good you found some pathological examples, but I’m at the end of my rope here.
You can use these examples and the other information you gathered so far and ask specifically how these size discrepancies can be explained and maybe mitigated. I suggest more specialized communities for this such as !linux@lemmy.ml, !linux@programming.dev, !linux@lemmy.world, !linux4noobs@programming.dev, !linux4noobs@lemmy.world, !linuxquestions@lemmy.zip.
sbeak@sopuli.xyz 9 hours ago
I have cross posted to a Linux community. Thank you so much for all your help :DDDD
sbeak@sopuli.xyz 10 hours ago
I’m assuming that Filelight count file size differently, and I will be trusting the result from dolphin more
sbeak@sopuli.xyz 10 hours ago
using -H with rsync doesn’t seem to do anything unfortunately…
sbeak@sopuli.xyz 10 hours ago
With Dolphin Godot directory is 1GB vs 5GB. Why is there a difference between filelight, dolphin, and du -sh? So weird
sbeak@sopuli.xyz 10 hours ago
it looks like much of the extra Godot bulk is in .git and .Godot directories
sbeak@sopuli.xyz 10 hours ago
Something about never knowing the time when one has two clocks
sbeak@sopuli.xyz 12 hours ago
Oh that’s actually a good idea. Thanks person! I will report back soon
sbeak@sopuli.xyz 10 hours ago
YIKES, I found that .local is around 30GB on my system ssd, over 50GB in the external SSD. Much of that is due to Steam and Kdenlive. I can try uninstalling Steam…