pooling media libraries - like distributed storage

Submitted ⁨⁨10⁩ ⁨months⁩ ago⁩ by ⁨suntzu@lemmy.dbzer0.com⁩ to ⁨selfhosted@lemmy.world⁩

I run a full media server, as well do a few friends. Now we had the idea to share our media libraries. In a first quick attempt we, mounted each other’s library folder via an smb share and imported those in jellyfin (all servers connected by VPN) Works quite well, but is kind of cumbersome the more people get in. I had the following idea: distributed storage, not as in redundancy, but more like mergerfs. Each “node” allocates a certain amount of storage, say node A, B and C provide 1TB each, these get fused into a singe mount that shows up as 3TB volume. If one node goes offline, the volume will only be 2TB and all files on the offline node will of course be unavailable.

Did a bit of research and found stuff like ceph,.glusterfs or seeweedfs, all of which I guess have a lot more functionality and thus are quite complicated and a little over my head. Do you do something like that or have any good ideas how to do that easily?

source

Comments

Sort:hotnew top

Vendetta9076@sh.itjust.works ⁨10⁩ ⁨months⁩ ago
I use Plex instead of jellyfin, but there’s the ability to just add a friends library and it pulls in. I thought Jellydin had that as well?

source
- originalucifer@moist.catsweat.com ⁨10⁩ ⁨months⁩ ago
  plex uses a centralized service for this kinda of nonsense. most of us are using standalone server products.
  
  this use case calls for either centralized storage (s3 bucket) or access mechanism(all them vpns) to distributed channels (ala plex)... but friends dont let friends use plex.
  
  im curious about ipfs as distributed file systems sound like a new kink i should have
  
  source
  - cm0002@lemmy.world ⁨10⁩ ⁨months⁩ ago
    
    but friends dont let friends use plex.
    
    I would love to get rid of Plex, but jellyfin failed the spouse test last summer and it never really liked my GDrive mount
    
    Plus, Plex clients are everywhere, so it’s all but guaranteed that whoever I decide to onboard is going to have something compatible. I’ve even had early smart TV’s from like 2013 with that weird Yahoo app store thing that had a Plex app that still worked even when the Netflix app didn’t lolol
    
    source
    -> View More Comments
  - density@kbin.social ⁨10⁩ ⁨months⁩ ago
    tell me why i shouldn't use plex as I'm always tempted by it whenever these threads come up and everyone who uses it is so happy.
    
    But free/libre is so much more delicious.
    
    But don't let the perfect be the enemy of the good.
    
    source
    -> View More Comments
  - Vendetta9076@sh.itjust.works ⁨10⁩ ⁨months⁩ ago
    Can you not just use a reverse proxy for your jeyllfin server and add multiple servers to the same client?
    
    source
    -> View More Comments
- lemann@lemmy.dbzer0.com ⁨10⁩ ⁨months⁩ ago
  I run Plex too, and indeed library sharing is built right in and ridiculously easy to set up
  
  source
Cooljimy84@lemmy.world ⁨10⁩ ⁨months⁩ ago
Another one to have a search for is IPFS.

source
- kpw@kbin.social ⁨10⁩ ⁨months⁩ ago
  All files stored on IPFS are public. It's also incredibly slow and inefficient. You would be better off using BitTorrent.
  
  source
zerodawn@leaf.dance ⁨10⁩ ⁨months⁩ ago
Likely not the solution you’re looking for but a buddy and i link a folder via syncthing and anything added to one side shows up on the other.

source
- suntzu@lemmy.dbzer0.com ⁨10⁩ ⁨months⁩ ago
  I use synching too, but it’s not what I’m looking for here.
  
  source
outcide@lemmy.world ⁨10⁩ ⁨months⁩ ago
Ceph, GlusterFS, and I suspect SeaweedFS (but I haven’t used it) expect high speed, low latency connections to their peers. So they won’t work well over the internet.

There’s some info floating around about using IPFS as the backend for Jellyfin, which in theory should allow you to share media between friends, but I haven’t tried it.

www.youtube.com/watch?v=PHujBhq4J9A

source
scrubbles@poptalk.scrubbles.tech ⁨10⁩ ⁨months⁩ ago
Seems to me the easiest solution would be each host a replica. Now that you can get 8TB for something like a hundred bucks this would be both faster and more redundant if one would fail

source
MSgtRedFox@infosec.pub ⁨10⁩ ⁨months⁩ ago
What’s the cumbersome part?

VPN? Mesh overlay VPN like tailscale/nebula mesh can do easy node add.

IPFS nodes might do the trick as mentioned.

source
- suntzu@lemmy.dbzer0.com ⁨10⁩ ⁨months⁩ ago
  On each node, map all the other nodes as smb, and configure all in jellyfin.
  
  It would be nicer to have one single mount.
  
  source
  - MSgtRedFox@infosec.pub ⁨10⁩ ⁨months⁩ ago
    What about using symlinks?
    
    You creat a directory /media. Mount shares there. Your media application scans /media and just finds media files.
    
    Still sucks because you have to mount each repo, /media/person1/movies, etc
    
    But you don’t have to reconfigure media app anymore.
    
    I don’t know what a pooled remote file system like what you’re wanting.
    
    source
uzay@infosec.pub ⁨10⁩ ⁨months⁩ ago
Why do you use SMB instead of just connecting to the different jellyfin servers directly via VPN?

source
- suntzu@lemmy.dbzer0.com ⁨10⁩ ⁨months⁩ ago
  One big shared media volume has multiple benefits, each server just have to deal with their own user management, no server switching or remembering if that one movie is of this or that Server…
  
  source
Decronym@lemmy.decronym.xyz ⁨10⁩ ⁨months⁩ ago
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

Fewer Letters More Letters

Git Popular version control system, primarily for code

Plex Brand of media server package

VPN Virtual Private Network

[Thread #399 for this sub, first seen 4th Jan 2024, 03:05] [FAQ] [Full list] [Contact] [Source code]

source
planish@sh.itjust.works ⁨10⁩ ⁨months⁩ ago
I think you can keep doing the SMB shares and use an overlay filesystem on top of those to basically stack them on top of each other, so that server1/dir1/file1.txt and server2/dir1/file2.txt and server3/dir1/file3.txt all show up in the same folder. I’m not sure how happy that is when one of the servers just isn’t there though.

Other than that you probably need some kind of fancy FUSE application to fake a filesystem that works the way you want. Maybe some kind of FUES-over-Git-Annex system exists that could do it already?

source

Fewer Letters	More Letters
Git	Popular version control system, primarily for code
Plex	Brand of media server package
VPN	Virtual Private Network