plantcv
plantcv copied to clipboard
danforthcenter
readbayer color channels are out of order
Describe the bug readbayer() is producing color images where the bands are out of order and image is rotated. I am actually getting a RGB image back (with bayertype 'BG') which means all the pcv functions are converting it to BGR when displaying, for example.
I tracked down cvtColor bug that seems to exist on and off in opencv from 2.4 -> 4.1 https://github.com/opencv/opencv/issues/4857
The last version on conda forge with the correct behavior is 3.4.2. (did not try 4.x series)
To Reproduce download https://github.com/danforthcenter/plantcv/files/4314758/Pats.Wheat.Root.Growth_002-0001_A1_2019-06-11_07-19-08.461_0.zip
import cv2
from plantcv import plantcv as pcv
filename='data/images/Pats Wheat Root Growth_002-0001_A1_2019-06-11_07-19-08.461_0.tif'
image_raw=cv2.imread(filename, -1)
img = cv2.cvtColor(image_raw, cv2.COLOR_BayerBG2BGR)
pcv.plot_image(img)
Expected behavior
or
even better rotated too which happens with opencv 3.4.2
Local environment (please complete the following information): windows 10, conda, pcv 3.10.1 and latest master. seems dependent on opencv >3.4.2
Additional context 2 possible solutions until upstream gets fixed:
-
limit opencv to 3.4.2
-
switch the color rendering to
I haven't looked at this from every angle yet but here's what I have so far.
Using OpenCV 3.4.10 I reproduce the incorrect first image above. In that version cv2.COLOR_BayerBG2BGR encodes code 46.
If I use the same Bayer filter we use on our LemnaTec camera (cv2.COLOR_BayerRG2BGR, which is code 48), I get the correct second image above (not rotated).
I ran through 3.4.2, 3.4.4, 3.4.7, 3.4.8, and 3.4.9 from conda-forge and 3.4.2.17, 3.4.4.19, 3.4.7.28, 3.4.8.29, 3.4.9.33, 3.4.10.37, 3.4.11.45 from PyPI and can confirm that with BG2BGR for 3.4.2 and 3.4.4 the color is correct but the images are rotated. From 3.4.7 onwards the images are the correct orientation but the color is incorrect.
I checked the value of each of the cv2.COLOR_Bayer* modes and they match across all versions and sources. Based on the thread I tested the idea that the the BG2BGR and BG2RGB were identical because BGR pointed to the wrong method but the images are different across all versions.
I'm wondering if the issue was in 3.4.2 and 3.4.4 and that the RG method is correct but looked wrong previously because of the potential bug?
oh! that's interesting. I potentially developed readbayer() with a buggy cv2. I don't know what version of opencv I was using then. did you go backwards from 3.4.2 ?
I was hoping to look back at the build logs to see exactly what version tests were running on but because we switched platforms I don't seem to have access to the old logs. But readbayer was added to release v3.0, which was about 22 months ago. On conda-forge, the opencv package was at 3.4.4 around that time. The PyPI version was in the 3.4.4 to 3.4.5 range.
oh good sleuthing! sounds like we should fix as if >=3.4.7 is correct.
I'm looking at the camera tech docs for my rgb camera and it is list as BayerGR8 as the format even though I was using BG. I don't remember if I looked this up and didn't understand why it didn't work (or just had no clue more generally) but it seems we should change the default in readbayer() to 'GR' to maintain the same visual behavior. I can test it this afternoon.
thanks for your help!
Ok, this is making sense. If I convert the color in 3.4.7 with COLOR_BayerRG2BGR (instead of BG2BGR) then I get the correct band order. that is, pcv.plot_image() shows true RGB. The difference is the rotation is off 90. In 3.4.7 the image is 90 ccw from what I was getting with 3.4.2. I can fix my roi locations if we think this is resolved. i'd suggest changing the default readbayer() to 'RG' so it is as we meant and setting the min opencv to 3.4.7.
Awesome! Changing the default and adding the version restriction to our requirements makes sense to me too.