Syncthing alternatives

Submitted ⁨⁨3⁩ ⁨weeks⁩ ago⁩ by ⁨ZeDoTelhado@lemmy.world⁩ to ⁨selfhosted@lemmy.world⁩

Hi,

As the title suggests: what are alternatives to syncthing that are basically fire and forget, works on multiple device types, and just focuses on file syncing?

I’ve had over the months the weirdest problems with syncthing, and lately I noticed some of my photos got corrupted, which is an absolute no no for me. I use syncthing currently as a easy automatic backup of documents, photos and other files, between my PCs and my phones (they all send only to the server. Folders are not shared with other devices).

source

Comments

Sort:hotnew top

ddash@lemmy.dbzer0.com ⁨3⁩ ⁨weeks⁩ ago
I know this is unsolicited advice, so apologies for that, but did you consider that the issue could also be your server hard drive failing? I guess not a possibility if its really only recently synced pictures. Just wanted to throw that in.

Regarding backing up photos, Immich works well for that but is also a lot more than just photo backup, so maybe not a preference for you.

source
- DragonSidedD@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago
  I’d echo this. Syncthing is widely used because it Just Works in so many weird situations. So if it is Not Working, swapping out Syncthing might actually make the problem harder to solve.
  
  source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  That is actually a good question, in this case the HDD is totally fine. Thing is, the photos only get corrupted very rarely (probably 1 every 3000 or so, and only sometimes) and often photos that got transfered several times to the phone (usually the problem is in the phone. On the desktop never had that)
  
  As for immich, I do know about of but seems to be a bit much for what I need (and only photos AFAIK)
  
  source
  - IanTwenty@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
    …could it be your phone’s storage is failing then?
    
    source
    -> View More Comments
avidamoeba@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago
That’s really weird. I’ve been using it for mobile-desktop-server-offsite sync for many years, with transfer sizes over 15TB, over WiFi, cellular, cable, fiber. I’ve never seen data corruption. Conflicts, sometimes. Permission issues, sometimes. Wiping something accidentally, sometimes. It’s even more weird because Syncthing performs computes hash values for the files it manages. I don’t know if it performs hash validation after copying remotely but if not, it can be forced manually which would tell you what’s fucked and be pulled from the source, if it still exists.

source
- halcyoncmdr@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  
  Dug into it, got into Memtest’s source code and discovered that the first pass is shorter on purpose so that it quickly flags obviously bad RAM. Apparently if you want to detect less obvious issues, you have to run multiple passes.
  
  I thought it was common knowledge that Memtest needed to be run for multiple passes to truly verify there are no issues. Seems that’s one of those things that stopped being passed down in the community over the years. Back when I was first learning about overclocking around 2005 that was emphasized HEAVILY, with the recommendation to run it at least overnight, and a minimum of 10 passes.
  
  source
  - avidamoeba@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago
    It’s kind of embarrassing because I used to work as a service technician at a popular computer store in the 2000s and Memtest86+ has been a standard fare of testing. I guess outside of OC, the shorter first pass truly was enough to spot has RAM in the vast majority of cases. Plus multichannel interactions were not nearly as prevalent in the DDR1/2/3 days. I recently installed 4 DIMMS for 128GB on an AM5 machine just to discover that the 5600 RAM only boots at 3600 in a 4-DIMM config, as per AMD’s docs. Could force it higher but without extra adjustment it can’t go beyond 4600 on this machine. Back in the day, different DIMMs, often with different chips worked in 2, 4-DIMM configs so long as they matched their JEDEC spec. backinmyday.gif
    
    source
    -> View More Comments
  - NullPointerException@programming.dev ⁨3⁩ ⁨weeks⁩ ago
    The software should inform the user to run atleast 10 passes in the UI
    
    source
- zorflieg@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  This post doesn’t benefit me at all but I love how long it is.
  
  source
  - avidamoeba@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago
    Let me tell you about diagnosing a reproducible crash on that 5950X system after swapping the RAM with verified good modules, which I only discovered because I decided to warm myself using Folding@home for a couple of cold days while the building was switching the central heating on.
    
    source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  That is some good info here. My HDD is totally fine (checked it very recently actually), as for the ram last time I checked was ok, but can check again to be sure
  
  source
  - avidamoeba@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago
    Check my edit.
    
    source
    -> View More Comments
autonomoususer@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

Is Syncthing my ideal backup application?¶

No. Syncthing is not a great backup application because all changes to your files (modifications, deletions, etc.) will be propagated to all your devices. You can enable versioning, but we encourage you to use other tools to keep your data safe from your (or our) mistakes.

docs.syncthing.net/users/faq.html#is-syncthing-my…

source
- Mio@feddit.nu ⁨3⁩ ⁨weeks⁩ ago
  This is why you should have file history versioning on. With backup this is a most.
  
  source
- Nimrod@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
  Would syncing your backup directory work? Like take snapshots of your system, dump the snapshots all in a single directory, and sync it to an off-site location?
  
  source
  - milicent_bystandr@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
    Kind of. That improves your backup safety, but doesn’t mitigate all the risks. E.g. if you accidentally delete everything from your backup directory, then all those deletions also happen on the sync’d one.
    
    source
  - autonomoususer@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
    No, re-read above.
    
    source
sorter_plainview@lemmy.today ⁨3⁩ ⁨weeks⁩ ago
First and foremost Syncthing is not a ‘backup’ utility. Using it for backup is not at all recommended. Especially if you are dealing with Android or Raspberry pi, because the way clock / time works in these systems are pretty weird and create sync conflicts. So don’t.

Now to the solution. For backup, use a proper backup solution like Kopia. Modern solutions support browsing the snapshots created as backups. Also creating periodic snapshots ensures better redundancy and better chance for disaster recovery.

Now if you will not use it for backup, take a look at ‘Round Sync’ available in F-Droid. It’s an application built around the execptionally good app, ‘rclone’. It is similar to Syncthing, but designed in a very different way. Also it is more difficult to configure to copy the files to PC.

I also wanted to mention that I have used Syncthing for many heavy lifting jobs and never faced issues with it. It is a feature complete app, with the philosophy of doing only one thing and doing it perfectly. So if you run into any issues, do reach out to forums or devs. They will definitely help you.

source
- milicent_bystandr@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
  Curious about your point about time conflicts. Doesn’t syncthing look at the change on your machine compared to the ‘canonical’ list also stored on your machine? So even if the timestamp is different, syncthing still detects the change, and the only problem is if the file is simultaneously modified on another machine before being propagated - which would be a conflict anyway.
  
  source
  - sorter_plainview@lemmy.today ⁨3⁩ ⁨weeks⁩ ago
    It didn’t work like that for me. I must admit I didn’t dig deep to clearly see what is the problem. So my setup had a Windows Pc, a Raspberry Pi 5, and an Android phone, sharing a folder which had notes.
    
    Whenever I save any changes in Windows machine, the android used gets updated without much issue, but the Raspberry Pi caused conflicts. When looked at the time stamps they were different and it looked to me like the Raspberry Pi 5 Syncthing is sending the old file as new one, because of the save time.
    
    It read somewhere the issue is with how time is handled in Rasberry Pi. So I disabled the Raspberry Pi Syncthing and went on, because that was not really needed.
    
    source
    -> View More Comments
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  From what I see, kopia is for the desktop. Unless I didn’t see something, it is not available for android, which is where more important to have backup in my case
  
  source
  - sorter_plainview@lemmy.today ⁨3⁩ ⁨weeks⁩ ago
    Aah my bad… I was half asleep. What I meant was use Round Sync / Syncthing to copy files to pc and then use Kopia to backup. Round Sync can do one direction copying, so source files are not corrupted.
    
    source
    -> View More Comments
ShortN0te@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago
Really surprised about this. I am using syncthing now for many years on various devices and never encountered issues with it. And also, file sync is not a backup solution.

source
- sxan@midwest.social ⁨3⁩ ⁨weeks⁩ ago
  Ditto.
  
  I get angry with SyncThing; don’t get me wrong. I really wish they’d add a per-file-type merge plugin capability, and I get far more sync conflicts than I care for. I get situations where a client on one computer stops (mostly, Android killing it) and it needs to be manually restarted.
  
  What I’ve never had it data corruption. It’s to the point where I implicitly trust that if SyncThing says it’s synced, I know it’s on the destination. It might be a stored as a sync conflict, but it’s there.
  
  source
Onomatopoeia@lemmy.cafe ⁨3⁩ ⁨weeks⁩ ago
I’ve not had files get corrupted, that’s strange, for sure.

Some options (but it really depends on what you’re trying to do):

Resilio Sync (cross-platform, battery eater on mobile, but has a useful feature - Selective Sync)

FolderSync on Android (can also be battery intense)

Native tools and scheduling on Linux and Windows.

Maybe a review of the setup is in order, what your devices are, what you’re trying to do. Sync may not be the tool for what you’re trying to do.

I use a combination of tools for different jobs and devices.

Generally, all files on mobile are synced to home using Syncthing-Fork. For example, on Android I have a sync job(“folder”) in ST for every root folder on the device. These get synced to home depending on conditions (e.g. DCIM syncs over any network, any power condition, so I don’t lose photos, while Backups and Download only sync on power over wifi, since they’re big and not critical).

Between desktop OS’s I largely use OS-specific tools, because I don’t really need sync there, but file copy and comparison on a schedule.

After 10 years and probably 1TB+ of sync, I haven’t lost any files via ST. Currently have 5 phones running jobs back to a server. I’ve even moved Syncthing from one server to another with no issues. With how ST works, I think file corruption is highly unlikely - it even handles in-use files safely.

source
ladfrombrad@lemdro.id ⁨3⁩ ⁨weeks⁩ ago
SMB (the protocol) works across devices and OS’s, and I use SMBSync2 to dump my photos to my NAS + off site

files.catbox.moe/rpqu4h.jpg

This has worked fine for me for years even thou the app isn’t on the Play Store anymore, and is fairly granular in regards to file types and where you wanna back them up

github.com/Sentaroh/SMBSync2

source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  This is an interesting one, will check. Thanks! I do have some smb stuff already on the machine, so this might be an easier solution
  
  source
  - ladfrombrad@lemdro.id ⁨3⁩ ⁨weeks⁩ ago
    I love it, and along with the folder they get dumped into being R/O means Immich (and by extension me…) can’t fugg up the timeline ;)
    
    source
the_wiz@feddit.org ⁨3⁩ ⁨weeks⁩ ago
I am using rsync triggered by cronjobs for this task now for… well… nearly forever.

source
- milicent_bystandr@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
  If you ask Syncthing how to do local sync (e.g. to an external HDD), the answer is, use the right tool for the job: Unison.
  
  If you ask Unison how to do certain things (directory timestamps is the one I miss), the answer is, use the right tool for the job: rsync.
  
  In the end, it all comes down to rsync.
  
  P.S. I’m actually gradually migrating up the chain from rsync, having used my own hand-built utility to make convenient rsync commands, but now using syncthing and Unison more.
  
  source
hera@feddit.uk ⁨3⁩ ⁨weeks⁩ ago
This seems very odd, I’ve been using syncthing for a good 5 years and never had a single file corrupt - might be worth opening an issue on their github

source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  I could try to do that, but I simply do not have reproducible steps that are certain to make the problem happen. I am a developer myself, and I absolutely despise when someone says ‘hey, something random happened the other day. I cannot say what are the steps, but it is there’ just to find out in the end nothing is there, or is simply not reproducible no matter what and for reasons that I might never find out
  
  source
  - hera@feddit.uk ⁨3⁩ ⁨weeks⁩ ago
    Someone might recognise your issue though and have suggestions, even if you can’t reproduce it exactly
    
    source
antihumanitarian@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
I had some similar and obscure corruption issues that wound up being a symptom of failing ram in a main server node. After that, only issues have been conflicts. So I’d suggest checking hardware health in addition to the ideas about backups vs sync.

source
Darkassassin07@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago
FolderSync selectively syncs files/folders from my phone back to my server via ssh. Some folders are on a schedule, some monitor for changes and sync immediately; most are just one-way, some are two-way (files added to the server will sync back to the phone as well as uploading data to the server). There’s even one that automatically drops files into paperless-ngx’ consume folder for automatic document importing.

From there BorgBackup makes a daily backup of the data, keeping historical backups for years with absolutely incredible efficiency. I currently have 21 backups of about ~550gb each. Borg stores this in 447gb of total disc space.

Image

source
NaibofTabr@infosec.pub ⁨3⁩ ⁨weeks⁩ ago
Ah, just one question - is your current Syncthing use internal to your home network, or does it sync remotely?

Because if you’re just having your mobile devices sync files when they get on your home wifi, it’s reasonably safe for that to be fire-and-forget, but if you’re syncing from public networks into private that really should require some more specific configuration and active control.

source
- Onomatopoeia@lemmy.cafe ⁨3⁩ ⁨weeks⁩ ago
  Syncthing runs encrypted anyway.
  
  source
  - NaibofTabr@infosec.pub ⁨3⁩ ⁨weeks⁩ ago
    Encrypting the connection is good, it means that no one should be able capture the data and read it - but my concern is more about the holes in the network boundary you have to create to establish the connection.
    
    My point of view is, that’s not something you want happening automatically, unless you manually configured it to do that yourself and you know exactly how it works, what it connects to and how it authenticates (and preferably have some kind of inbound/outbound traffic monitoring for that connection).
    
    source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  Great question. I forgot to mention but at this point syncthing is only accessible on my WiFi at home. No VPN to it and no remote locations.
  
  source
SanPe_@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
I use roundsync for Camera upload. Works well.

source
johntash@eviltoast.org ⁨3⁩ ⁨weeks⁩ ago
What about Nextcloud? It’s heavier than syncthing, but would be an alternative.

source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  From what I tried, next cloud just has son Mich clutter that makes the syncing part a bit of a slog. If that was rock solid with OK speeds, that would be nice. But there is somich stuff with it that I don’t care and tries to enforce (like users and user groups) that I find just too much
  
  source
Xanza@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
rclone.

source
- sxan@midwest.social ⁨3⁩ ⁨weeks⁩ ago
  How is rclone fire and forget? You have you manually run it every sync, right?
  
  source
  - Xanza@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
    You can set it up as a service to monitor a remote directory. I have it running using nssm on windows on my seedbox. I’ll torrent something, give it a while and boom, it’s on my NAS without doing anything.
    
    Same concept here. So yeah, set it once, and forget about it.
    
    source
    -> View More Comments
irmadlad@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
How about Unison, Resilio, or Seafile ?

source
- ZeDoTelhado@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  Never tried unison or resilio can check. As for seafile, that is what I had before. At some point I realized I was getting several issues with desktop mostly, and the storage was only accessible from seafile wich in my case I am not OK with. Mostly was the inconsistencies between oses
  
  source
  - irmadlad@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
    There is a dockerized version of GoodSynch, tho I’ve only used the windows version, so I can’t really vouch for it. Might be something worth looking at. I chime in with the others here in that I use Synchthing and I’ve never had any issues with corrupt files, but I can understand how that would be unacceptable.
    
    source