r5 icon indicating copy to clipboard operation
r5 copied to clipboard

Add Bucket/Folder abstraction to FileStorage

Open trevorgerhardt opened this issue 4 years ago • 0 comments

We currently need to pass a String bucket almost everywhere that we use and instance of FileStorage. We should consider creating a simple Bucket abstraction that wraps the FileStorage with immutable bucket field that is automatically used in all the standard FileStorage operations. This would allow us to drop most usage of the FileStorageKey.

For example, the OSMCache constructor:

https://github.com/conveyal/r5/blob/e2843f735d8a6d0d336446951ceceb6ddea6a9c3/src/main/java/com/conveyal/r5/streets/OSMCache.java#L32-L35

would become

public OSMCache(Bucket bundleBucket) {
  this.bundleBucket = bundleBucket;
}

and usage would go from

https://github.com/conveyal/r5/blob/e2843f735d8a6d0d336446951ceceb6ddea6a9c3/src/main/java/com/conveyal/r5/streets/OSMCache.java#L45-L53

to

public String getKey (String id) { 
     return cleanId(id) + ".pbf"; 
 } 
  
 public OSM get (String id) { 
     try { 
         return osmCache.get(id, () -> { 
             File osmFile = bucket.getFile(getKey(id));

The difference seems minor, but we use FileStorage fairly frequently that compounded across our code base it should simplify things quite a bit, lots of passing to constructors and re-storing a single variable instead of two to get the same effect.

Additionally, we could create generator functions that return a Function<String, File> getFile and BiConsumer<String, File> moveIntoStorage so that the individual requirements of each usage of FileStorage is more explicit. For example, the above would turn into:

public OSMCache(BiConsumer<String, File> getFile) {
  this.getFile = getFile;
}
...
 public OSM get (String id) { 
     try { 
         return osmCache.get(id, () -> { 
             File osmFile = getFile.accept(getKey(id));

trevorgerhardt avatar Feb 23 '21 22:02 trevorgerhardt