Android location saving after restart part 2

[troach-sf]

Safe Paths is not saving locations after device restart until the app is opened. Part 1 of the issue was here: #523. There was a react library bug that prevented the geolocation service from restarting upon a phone reboot.

This issue is for part 2, which is that the Safe Paths geolocation listener no longer exists after a device restart.

Version

• Type of phone: Android (Solution will impact iPhone)
• OS Version: All
• Model: All

Steps to reproduce

Take note of the locations inside RKStorage dataset (async-storage). Restart the application and do not open Safe Paths. No additional locations are logged in RKStorage until Safe Paths opens.

Expected Behavior

When I restart my device, I expect new locations logged in RKStorage before opening Safe Paths application.

Solution

react-native-background-geolocation has its own sqlite database of lat/long/timestamp with additional information. We can set a max row count of 8,064 which is 1 row / every 5 minutes / for 28 days.

We can then get list of locations by calling getLocations (https://github.com/mauron85/react-native-background-geolocation#getlocationssuccess-fail). We can then further filter on timestamp to ensure we only keep 28 days of data in the list.

There is no need to store locations in async-storage. At this point, it is only duplicating data that already exists in the react-native-background-geolocation database. This will be far more efficient, as the current method of adding a location in async-storage involves pulling the entire list of locations, appending a new location in an array, and then re-saving a json blob to sqlite.

The listener will no longer be required to store locations, and can be repurposed just for logging.

[troach-sf]

Async-Storage database. App installed, device restarted shortly after opening. 1 location saved.

[
  {
    "latitude": 34.24109059012831,
    "longitude": -87.95186424460518,
    "time": 1586979689990
  }
]

react-native-background-geolocation database. Same scenario as above with 2 additional locations tracked after restart.

[
  {
    "_id": 1,
    "time": 1586979689990,
    "accuracy": 1.27143120765686,
    "speed": 0.296305745840073,
    "bearing": 0,
    "altitude": 42.7420883178711,
    "latitude": 34.2410905901283,
    "longitude": -87.9518642446052,
    "radius": 0,
    "has_accuracy": 1,
    "has_speed": 1,
    "has_bearing": 0,
    "has_altitude": 1,
    "has_radius": 0,
    "provider": "network",
    "service_provider": 0,
    "valid": 2,
    "batch_start": "",
    "mock_flags": 11
  },
  {
    "_id": 2,
    "time": 1586979778114,
    "accuracy": 3.03377676010132,
    "speed": 0.685454130172729,
    "bearing": 0,
    "altitude": 42.0690498352051,
    "latitude": 29.585002832023,
    "longitude": -82.9246184489833,
    "radius": 0,
    "has_accuracy": 1,
    "has_speed": 1,
    "has_bearing": 0,
    "has_altitude": 1,
    "has_radius": 0,
    "provider": "network",
    "service_provider": 0,
    "valid": 2,
    "batch_start": "",
    "mock_flags": 11
  },
  {
    "_id": 3,
    "time": 1586979841314,
    "accuracy": 1.15071249008179,
    "speed": 0.426103919744492,
    "bearing": 0,
    "altitude": 46.5179405212402,
    "latitude": 30.6268530438688,
    "longitude": -92.4966430100099,
    "radius": 0,
    "has_accuracy": 1,
    "has_speed": 1,
    "has_bearing": 0,
    "has_altitude": 1,
    "has_radius": 0,
    "provider": "gps",
    "service_provider": 0,
    "valid": 2,
    "batch_start": "",
    "mock_flags": 11
  }
]```

[troach-sf]

I plan on making a few fundamental changes I have discussed with my team, and I want to make sure the solution is agreed upon by key stakeholders.

Current Location Tracking:

1. Locations are Stored in Async-Storage, while also being retained but unused in react-native-background-geolocation sqlite database.
2. If there are gaps in data, backfilled locations are applied if possible. Once these backfilled locations are created, there are no flags indicating if a location is real or backfilled.
3. Google Takeout imports location data into the Async-Storage dataset. Once the takeout data has been imported, it is not possible to determine if a location is from takeout, or the device.

My proposal:

1. Only store accurate and real captured locations in the sqlite database that is created by react-native-background-geolocation. We can set the maxCount = 8,064, which is 28 days worth of 5 minute location intervals. Since we need to fork the library anyway, I may add better sql bindings so that we can be specific to only keep 28 days, etc. Until then, we can filter out possible invalid dates after pulling the array from sql.
2. Google Takeout data stays in its own table or key/value pair as currently implemented.
3. When we need to act on data, we can merge datasets as required (Real + takeout), fuzzy data for more anonymity when reporting is implemented, etc, backfill data, etc. Keeping the sqlite database the record of truth for real locations, allows us to be more agile in tweaking our export options, without compromising high-accuracy if needed.

Future alternative proposal:
If we determine that we would like a merged dataset from all possible sources, we can do so with a column of record type: real, google, etc. I think this is overkill now, but could make things easier if we implement many import providers.

[tstirrat]

+1 on using data that is already there and more accurate.
+1 on not merging raw data sources so that we know where data came from

Will give final decision to @tremblerz but I support this.

[tremblerz]

Definitely like the idea of not merging Google's takeout data right away. Especially given that it could come either from Google Places or Google Location history.
Actions to be performed during export is also a great idea.
One question -

1. There was some discussion that we only wanna store the geolocation data in an encrypted format even on device, will db be a good idea in that case as we can't perform standard queries?

[troach-sf]

Unfinished and not tested, but updating this ticket with info on where the work is being done. I'll be done with this ticket by end of day monday.

https://github.com/safe-paths-contrib/covid-safe-paths/tree/feature/sqlite-location-tracking

[tstirrat]

Closed with #788 ( = #747)