large_image icon indicating copy to clipboard operation
large_image copied to clipboard

Improved image creation

Open manthey opened this issue 1 year ago • 4 comments

Currently, the vips source adds a new method that, when used with addTile, can take a series of images (PIL or numpy or vips) and output a file in a few formats. The addTile method species where in the single frame image the new data is located and allows for masking the data as it is added.

We want to expand this capability:

  • [x] Support arbitrary data axes to have multiple frames. The base image can have different sample depths (currently L, LA, RGB, RGBA, but it would be nice to support arbitrary hyperspectal sample channels OR indexed data). Frames could be arbitrary OR could have specific axes: C (channel, which should have channel names as optional metadata), Z (vertical Z stack), T (time), XY (also called P or P and Q for physical sample position). We don't currently handle other axes in any source, but having arbitrary axes would make reading things like netcdf more obvious.
  • [x] Support arbitrary numpy dtypes for the data
  • [x] Support a variety of inputs for data and masks: PIL, numpy arrays, vips images
  • [x] Set a variety of metadata: pixel scale (see #1482).
  • [x] Set image description (see #1483)
  • [x] Set channel names (#1484)
  • [x] Support cropping the data.
  • [ ] Support padding the data to different sizes. We might want to specify a default color for all channels.
  • [ ] Add data at different scales (e.g., a lower resolutions from the base). This could do a nearest neighbor mapping to the base resolution, but we might want to eventually add an option for how the data is scaled. The general form of this would be to accept an affine transform.
  • [ ] For geospatial data, specify a base projection and corner locations in some manner, and add an optional different projection when adding data.
  • [x] Have entry-point based file output plugins for this.
  • [x] Add associated images (#1485)
  • [ ] Add arbitrary additional metadata.
  • [ ] Allow new images to have a optional path so that they could be created over multiple program runs. This would require having a close method (preferably implicit) that saves enough state to resume image creation (#1487)

One method of storing this data internally would be a hdf5 file via the h5py module. We could make a new hdf5 source (limited to just files we create) to allow reading the data as it is being created.

We always want the property that the last data added "wins", overwriting any existing data at its location. If the added data has a mask, it should only change the appropriate locations. If it has an alpha channel, this should be applied as it is added. As part of this, we may want to have a default alpha channel that is transparent until any data is added. Note that if data is added to the same location multiple times with alpha values that are 0 < a < 1, we need to make sure that they mix in the expected proportion.

As a starting point, we should create tests that will exercise the existing features and then expand the tests as we add features.

manthey avatar Nov 29 '22 16:11 manthey