loam icon indicating copy to clipboard operation
loam copied to clipboard

Remote COG support?

Open drwelby opened this issue 3 years ago • 3 comments

Can this read COGs via vsis3? The gdal.js examples seem to all be local files.

drwelby avatar Jul 28 '21 16:07 drwelby

Good question! The answer is "kind of". vsis3 support requires libcurl to be compiled into the bundle, and currently that is not done, so you can't run loam.open('/vsis3/bucket_name/path/to/cog.tif) as things stand.

Emscripten provides a lazy file wrapper that can be used to wrap a remote file, and I've tested it out and it issues ranged reads as you would hope. However, it reads in fixed 1MB chunks, so there's a lot of efficiency lost because 1MB is rarely the ideal size for reading in the appropriate parts of the COG, so it either makes too many requests, or needlessly loads extra data. For that reason, I haven't added support for remote files directly to Loam, but it should be an easy change if it would be useful for someone.

The other two options for accomplishing this are

  1. Compile in libcurl support. This would increase the size of the downloaded WebAssembly. I don't know by how much, but as it's already 2.2MB compressed, I would prefer not to take this route. It would also make the build process quite a bit more complex, and I'm not totally certain that libcurl would function properly from WebAssembly, so there is some potential risk here.
  2. Make a PR to GDAL so that it can use Emscripten's fetch API as an alternative to libcurl when being compiled by emcc. This is my preferred option because it should limit bundle size and provide the smoothest integration between gdal.js and the browser's built-in HTTP request capabilities.

Can you tell me more about your use case? This is a high-priority feature for me because lots of people are asking for it, but as you can see, the preferred solution is quite a bit of a lift, so I'm happy to help brainstorm workarounds. Alternatively, if you have the expertise necessary to implement any of these solutions I would be very happy to review a PR!

ddohler avatar Jul 28 '21 18:07 ddohler

My use case is the standard "view a COG and metadata in the browser" use case that's handled by geotiff.js, but wasn't sure if this was something gdal.js was able to replace yet.

Your solution #2 sounds like the way to go, but unfortunately I don't have the technical ability to be any help implementing it.

drwelby avatar Jul 28 '21 18:07 drwelby

Okay, thanks! I will add this to the stack of requests for this feature. 🙂 The good news is that I'm close to finishing up the current set of features I'm working on, which mostly relate to developer experience, and this will be my next focus. It's not necessarily going to happen quickly, but I am planning to start working on this soon.

ddohler avatar Jul 28 '21 18:07 ddohler