Comment on Introducing reitti: a selfhosted alternative to Google Timeline
danielgraf@discuss.tchncs.de 3 weeks agoThanks for the information. I will try to recreate it locally. In my testing I used a 600MB file and this took maybe 2 hours to process on my server. It is one of these ryzen 7 5825U. Since Reitti tries to do these analysis on multiple cores we start it with 4 to 16 Threads when processing. But the stay detection breaks when doing it that way, so it is locking per user to handle that. If now one of them takes a long time the others will break eventually. They will get resheduled 3 times until rabbitmq gives up.
On what type of system do you run it?
I will add some switches so it is configurable how many threads are opened and add some log statements to print out the duration it took for a single step.
ada@piefed.blahaj.zone 3 weeks ago
i7-8700 with 64GB of RAM
danielgraf@discuss.tchncs.de 3 weeks ago
Hmm, I had hoped you say something like a Raspberry PI :D
But this should be enough to have it processed in a reasonable time. What I do not understand in the moment is that the filesize should not affect it in any way. When importing it 100 Geopoints are bundled, send to RabbitMQ. From there we retrieve them, do some filtering and save them in the database. Then actually nothing happens anymore until the next processing run is triggered.
But this than works with the PostGis DB and not with the file anymore. So the culprit should be there somewhere. I will try to insert some fake data into mine and see how long it takes if i double my location points.
ada@piefed.blahaj.zone 3 weeks ago
I was also trying to set up GPSLogger whilst it was crunching through the backlog, and I manually transferred a file from that app before I had autologging configured. Not sure if that could have done it?
The times don't overlap, as the takeout file is only up until 2023
danielgraf@discuss.tchncs.de 3 weeks ago
Thanks again for the follow-up. It is something I can investigate. I doubt that it is somehow related, but who knows. 🤷