Comment on Introducing reitti: a selfhosted alternative to Google Timeline
danielgraf@discuss.tchncs.de 1 day agoThis process is not triggered by any external events.
Every ten minutes, an internal background job activates. Its function is to scan the database for any RawLocationPoints that haven’t been processed yet. These unprocessed points are then batched into groups of 100, and each batch is sent as a message to be consumed by the stay-detection-queue. This process naturally adds to the workload of that queue.
However, if no new location data is being ingested, once all RawLocationPoints have been processed and their respective flags set, the stay-detection-queue should eventually clear, and the system should return to a idle state. I’m still puzzled as to why this initial queue (stay-detection-queue) is exhibiting such slow performance for you, as it’s typically one of the faster steps.
ada@piefed.blahaj.zone 9 hours ago
Well, the last update seems to have cleared the queue, and all of my history from that 10 year import now shows, with trips and places identified!
But now, it's having issues with importing the new google format import. I've got a 34MB file there that goes back to 2017, and this data says that it has imported, but then never appears in my history.
If it's relevant, there is overlap in the data, as my 10 year takeout import went up to 2023, and my "new format" import starts in 2017 and went a couple of days ago. I changed my google account in 2017, but logged in to both on my phone simultaneously, so I was accruing location data on both accounts at the same time for a while before I turned it off on my old account.
danielgraf@discuss.tchncs.de 3 hours ago
I am glad it worked out for you in importing the first file. I am still puzzled why it took that long.
For the new format, did you have Android or iOS? With the timeline export from Google Maps on iOS, we can not do anything at the moment because there is actually no raw data in it. Only information like you stayed in this timeframe at that point and you traveled between these points. It’s actually a little bit funny that it aggregates to the same data Reitti uses in the end.
If you are on Android, it could also be a bug when importing that file. I only had a small one from one of my accounts to test. If you mind creating a bug report, I will have a look. If you do not want to attach the export file there, feel free to send it to daniel@dedicatedcode.com. I will have a look at it then privately. No problem.
For the overlap in exports, it depends. If the points are the same, meaning they have the same timestamp, then Reitti will discard them. If not, they will be handled like every other new data and will end in recalculating visits and trips around that particular time.
ada@piefed.blahaj.zone 2 hours ago
I'm on android. I'll raise a bug report.