600MB. Would that require changes to Jellyfin? They’re on the path to introducing Postgres support.
Comment on Improve very slow library scans on Jellyfin 10.11 / 12 on spinning media
zr0@lemmy.dbzer0.com 1 day ago
How large is the database? Because there is a way to fully load SQLite into memory, which would speed everything up drastically for a scan. At the end of the scan, you could just write back everything to the disk, then it is just one blob of sequential write, which is fast enough on spinning rust.
avidamoeba@lemmy.ca 1 day ago zr0@lemmy.dbzer0.com 1 day ago
I just checked the code and unfortunately, they do not support loading the SQLite file into memory yet. Even worse, the change would be substantially. It appears they currently abuse the database connection a bit. They basically create a new connection for each operation. This does not matter if you can read/write in parallel, but that’s only the case with NVMe and certain drivers. So all SATA (SSD and HDD) suffer from the current mechanic. So just loading it into memory is not going to solve the issue. Jellyfin needs to change architecturally, so that a database connection is reused. I don’t know why or who implemented the current logic, but it is clearly done by someone not understanding how databases work.
Depending on the mood and time, I will come up with a solution that is solid. For now, you are stuck with the current logic.
- avidamoeba@lemmy.ca 23 hours ago
Thanks for looking into it. As far as I read they moved to an ORM with 10.11 as a prereq to supporting other db engines. I’m guessing the db connection code would be undergoing major changes when they do that so I wouldn’t rewrite the current implementation. I’d either follow that work and help them get it right or rework the implementation after they’re done. We’ll just suffer through the mean time. 😄
lyralycan@sh.itjust.works 1 day ago
Interesting, I figured people would use MySQL instead if possible.