dive icon indicating copy to clipboard operation
dive copied to clipboard

Add Girder Large Image and Tiled Image Support

Open BryonLewis opened this issue 3 years ago • 0 comments

resolves #1253

Features/Code Implemented:

  • Installs girder-large-image and dependencies - Uses a series of wheels, I couldn't find the poetry equivalent of --find-links for pip install. There is this PR (https://github.com/python-poetry/poetry/pull/1511), but it seems the still preferred way is to use direct links to the wheels.
  • Implements a new DatasetType: 'large-image' - automatically converts NTFs, NITFs, tiffs and R# files to large image data. This new type can be uploaded using the data importer
  • LargeImageAnnotator.vue - new viewer which loads and displays large-image. Required some changes to the useMediaAnnotator loading process to work properly. Also, have some commented-out code to enable an Openstreetmaps background on large images that have geospatial images. The problem with that is I didn't implement a transform from local image-space coordinates to global (lat/lons), so commenting it out will cause annotations to not work.
  • Handles the uploading of large image new file datasets as a large-image type in the client
  • Disables running of pipelines/training on large-image datasets in the viewer and the data browser
  • If a regular PNG/JPG image is larger than 8k in any dimension it will notify the user and suggest they convert to large-image. This was chosen because it's a typical max texture size for clients. Video Cards that aren't Nvidia Quadro-level workstations have lower single-texture resolutions that can be rendered. We use a single texture for the image in geoJS so this will automatically convert it to a large image so it can be displayed in the client.
  • I fixed an issue that was bugging me where when loading a single image the UI for the timeline wouldn't look right. I updated it so that tracks and detection counts span the entire width of the timeline if there is one image. This makes more sense when utilizing large-images where there may be only a single image.
  • Used the layer-swapping method of GeoJS for traversing through sequential large images. It loads up one image and then prefetches the next frame in the background so that playback can be a bit smoother. You aren't going to get 10fps but being able to do around 1fps depending on zoom level and data size helps.

Testing Procedure:

Testing Dataset zip file(https://viame.kitware.com/api/v1/item/63481282828794fdd97cc03c/download) - warning it is ~1.8GB.

Standard Large Image Testing:

  • do a full docker-compose build --no-cache to build the system with the new poetry dependencies
  • Test the new uploading of large images. Right now that includes these formats: https://github.com/Kitware/dive/pull/1255/files#diff-a2265279f516b5cd723f8575c4e1133065c2e5a253d9a38e0abe442619b8a13bR29-R44. The R# formats are used to display some level information for nitfs I've seen in the past. The one thing I don't have a good way to determine is the difference between a non-pyramidal tiff vs a regular tiff. For this first test I would use of the single files in my test package like Seattle or Anchorage.
  • Let me know if you know of a better term for this button: image
  • After loading the image check that you can see it and create/save annotations.
  • Confirm in the localhost:8010/girder interface that the metadata is set correctly and that the type is 'large-image'
  • Pipelines should be disabled both in the Annotation viewer and the data browser when large-image datatypes are selected.
  • Note specifically the change in the timeline/event viewer for annotations. Before single frame datasets wouldn't display the detection count or the events in the viewer. The new update view shows them as being across the full frame

Sequenced Large Image Testing:

  • Within the sample dataset folder I have a subfolder called 'Sequence' which has a sequence of NITFs that can be loaded into the system.
  • This should be similar to image-sequence except that transitioning between frames is going to take much longer. There is buffer of one frame look ahead to ease the transition but when you randomly seek around it will require some loading and the loading indicator should be displayed showing that.

Auto Conversion of Large JPG/TIF/PNG Images:

  • Within the testing zip is a folder called 'large_jpgs' which includes several JPG files that are larger than 8k in either width or height. On regular graphics cards these won't render properly in the annotation viewer.
  • Import these as a standard image first and then launch the dataset.
  • A pop-up asking to convert it to a large-image format will be shown. Accepting that will kick off a quick job which will convert the type of the dataset to large-image and allow viewing for all people these images.

TODOs:

  • [ ] Integration testing for uploading and importing large images. Will need to update the zip file containing integration test source data as well as update the tests themselves to handle the large image.
  • [ ] Documentation Updating: Supported Formats and types, some information about Auto Conversion of large JPGS/PNGS
  • [ ] Pipeline testing: For now I've disabled pipelines on all large-images even though theoretically large JPGS/PNGS and non-tiled TIFFs should work.

BryonLewis avatar Jun 01 '22 19:06 BryonLewis