metrics-mvp icon indicating copy to clipboard operation
metrics-mvp copied to clipboard

Avoid concurrency errors when saving cache files

Open youngj opened this issue 5 years ago • 0 comments

Some backend Python modules have functions that write cache files in the backend/data directory. However, in production it's possible that multiple Python processes could try to write the same cache file at the same time, which could cause invalid data to be stored in the file. This issue could be prevented if the cache files were written atomically, such as by writing a temporary file and then renaming it to the desired file name.

These methods include arrival_history.get_by_date, precomputed_stats.get_precomputed_stats, routeconfig.get_route_list, timetables.get_data_by_date_key, and timetables.get_date_keys .

Perhaps the similar logic in these functions could also be refactored to reduce duplicated code.

youngj avatar Feb 20 '20 03:02 youngj