r5 icon indicating copy to clipboard operation
r5 copied to clipboard

Remove MongoJack and consolidate MongoDB utils

Open trevorgerhardt opened this issue 2 years ago • 0 comments

A process that started many years ago, this removes the MongoJack dependency and completes the incremental replacement of the Persistence module and its collections with the AnalysisDB and AnalysisCollection types.

In light of a possible database switch in the near future, it seemed more straightforward to continue to use String based IDs everywhere, instead of switching to a MongoDB specific ObjectID type.

Our average collection already uses String IDs, but this will require a migration for aggregation areas, data sources, and data groups.

Other notes:

  • Switched from arrays to Lists in MongoDB model classes.
  • Added a DbResultWriter that sets completed: true on regional analyses when they are complete.
  • Made two OSMCache methods static (cleanId and getKey) so that the BundleController no longer requires it as a component dependency.
  • Updated the MongoDB driver from v3.11.0 to v4.7.2. MongoDB currently provides documentation for v4.3 to v4.8.
  • Removed unused HTTP endpoints (and their handlers) in the AggregationAreaController, BundleController, DataSourceController, and RegionalAnalysisController.
  • Refactored BundleController's created method, breaking it up into smaller steps.
  • Refactored RegionalAnalysisController's create method. Some benefits:
    • The FileStorage and AnalysisDB components no longer need to be injected through the Broker to the MultiOriginAssembler. Result writers are created in the controller with what they need.
    • Moved the RegionalTask creation into the AnalysisRequest object, where much of the logic already lived. More refactoring can certainly be done here, but it is a good first step.
    • Removed any usage of the RegionalTask stored on a RegionalAnalysis. This means that we only need to serialize it into MongoDB, we don't need to handle deserializing it.

This will resolve https://github.com/conveyal/r5/issues/758.

trevorgerhardt avatar Nov 08 '22 09:11 trevorgerhardt