imageprocessing icon indicating copy to clipboard operation
imageprocessing copied to clipboard

Using the Batch Processing for each corrected image instead of stacks

Open Prosperoux opened this issue 4 years ago • 12 comments

Hello dear community, I have followed correctly the tutorials in each steps. It works for a own single capture which is correct. Now i try to use the batch processing, but i would like to get each image corrected in a loop, but not using the stacks. How can i modify the code to get these results ? I am newbie in python and associated library, it's hard now to understand how to proceed. Can someone help me ? Thanks !

Prosperoux avatar Oct 13 '20 20:10 Prosperoux

It's possible what you're looking for is already proposed in the pull request here: https://github.com/micasense/imageprocessing/pull/110

There were a couple of small issues with that which is why it has not been merged yet; maybe you can check out that branch and test if it does what you need?

poynting avatar Oct 15 '20 16:10 poynting

Great ! Thanks. The modified capture.py with function capture.save_bands_in_separate_file from the code "Align images and save..." works. I ran with "max_alignment_iterations = 100000" and "pyramid_levels = 3" after several trials to get a correct alignment (without the rig relatives because i own an old Rededge3). Issues now are :

  1. In the "stacks" directory (individual bands) it displays file names like this : image and so on.
  2. even if the log.csv is correct when running the "Extract Metadata from list and save to log.csv" , the running of "Use Exiftool from the command line to write metadata to images" doesn't write any metadata in the images. What could i change for those issues ?

Prosperoux avatar Oct 15 '20 22:10 Prosperoux

You need to update the next cell "Extract Metadata from Captures list and save to log.csv" to write out a line for each image, instead of each capture. You may also need to add a column to designate the band, but that will depend on how the downstream processing differentiates bands.

for capture in imgset.captures:
    for image in capture.images:
        outputFilename = capture.uuid+'_'+image.band_index+'.tif'
        ...

poynting avatar Oct 15 '20 23:10 poynting

The paths and names in the csv file have to match the image paths exactly. I've had issues with things like spaces in the paths or .. in the path.

If you run the command that the last cell creates in your command prompt, sometimes it will give helpful information for debugging.

poynting avatar Oct 15 '20 23:10 poynting

You need to update the next cell "Extract Metadata from Captures list and save to log.csv" to write out a line for each image, instead of each capture. You may also need to add a column to designate the band, but that will depend on how the downstream processing differentiates bands.

for capture in imgset.captures:
    for image in capture.images:
        outputFilename = capture.uuid+'_'+image.band_index+'.tif'
        ...

Ok. I tried the proposed loop, unfortunately creates a "string" error... I finally left the original beggining, and remove the '.tif' as mentionned in the screenshot below 👍 image Results in new directory are like this, it's correct : image Running the cell for create log.csv (original code) return a file like this : image Thus, each exiftool extracts correct each metadatas from the Captures. The issue, in the final cell is for exif re-insertion from Captures to Images. There is certainly many ways to do that, using a kind of loop. I will look forward at Exiftool options list.

Prosperoux avatar Oct 16 '20 15:10 Prosperoux

Right, so you can see in the exiftool log the filename is wrong. Your single-image filenames have _N on the end. You may need something like

outputFilename = capture.uuid+'_'+str(image.band_index)+'.tif'

poynting avatar Oct 16 '20 19:10 poynting

But the code you are showing above is from the saving step. The next cell which has the header "Extract Metadata from Captures list and save to log.csv" is what does the creation of the file that exiftool needs. So you need to also modify that cell to update the filenames and create a new line for each image instead of each capture.

poynting avatar Oct 16 '20 20:10 poynting

Ok. Finally, that works fine this way : image And the log.csv is : image Now, when running final cell "Use Exiftool from the command line to write metadata to images", that doesn't write anything. I suppose exiftool should be run on the prompt command line ? I get error also, will see tomorrow. Thanks for helping it's nearly finished now. By the way, i have noticed that when increasing the pyramid level (3 in my case) the size of pictures decrease .

Prosperoux avatar Oct 16 '20 22:10 Prosperoux

That looks more promising. When I originally wrote it, the command would run properly from the notebook, but more recently I need to copy/paste it to the command line. I haven't dug into that problem, as it is inconsistent across different platforms.

poynting avatar Oct 16 '20 22:10 poynting

I have generated a code for processing all dataset. You can check in https://github.com/tuncaemre/Micasense-Altum-Image-Processing

tuncaemre avatar Dec 23 '20 03:12 tuncaemre

Any updates @Prosperoux ? Can we close this issue?

poynting avatar Apr 19 '21 05:04 poynting

Is there a way for me to skip the alignment altogether? I am trying to adapt this for use with rasmusfenger's Sequoia processing fork which will not align the bands, however, I can successfully calibrate a single capture using a combination that repository's tutorial and by manually supplying a calibration panel, pixel coordinates, and reflectance factors per the first tutorial in this repository.

jnederend avatar Sep 22 '21 20:09 jnederend