SafeTrace icon indicating copy to clipboard operation
SafeTrace copied to clipboard

Improve data structure

Open cankisagun opened this issue 4 years ago • 1 comments

Data is overwritten each time a user submits data - this can be improved but is hard. Currently user data is stored inside the enclave as a Rust HashMap indexed by the userId as its key, and the array of locations as its associated data. so everytime a new dataset is added to HashMap overwrites whatever prior entry was there for that key. Improving on this is hard because one would need to find data overlaps in terms of space and time with prior entries and do a proper merge.

cankisagun avatar Apr 08 '20 17:04 cankisagun

Heh, I actually just had thoughts for this -- I'd recommend as a first pass splitting up the time/location stamps into 0.05 degree lat long square chunks in location and 15 minute increments of time, with user-data hashmaps stored at each point in that latXlongXts array (I'm assuming Rust can handle sparse vectors in O(1) with some sort of standard hashmap modification, otherwise this scales beautifully but has a constant cost of something like O(10^10) hashmaps created) (with 4 per 0.1 degree chunk, so you have overlap)

FishmanL avatar Apr 12 '20 18:04 FishmanL