Manual selection of morph amount required

[gabrielburnworth]

The detect_weeds function to process an image uses a default or selected morph amount. This value usually needs to be optimized for each image to:

• Avoid marking multiple weeds close to each other as one weed with a center between the two
• Avoid marking a void inside a weed as a separate new weed

Sometimes there is not an optimum solution using the current approach of a closing morphology (as described here).

Multiple plant error:

Void false positive:

Both errors:

[roryaronson]

Have you tried a combination of opening and closing? It seems like it would be useful to do a series of processes, each optimized for a specific purpose. For example:

1. erosion to remove noise
2. erosion to shrink the small "bridges" between multiple plants and effectively sever them from each other
3. dilation to bring the plants back to normal size (noise and "bridges" will not come back)
4. dilation to fill in holes

It seems like continuing to erode and dilate the shapes would "smooth" them. I'm guessing that if many (10? 25?) erosions and dilations were done back to back then all the outlines would turn into perfectly discrete circles.

[roryaronson]

Iterating through a closing morph helps reduce internal voids. After too many iterations though it starts to combine actually separate morphologies. See output of iterating 1, 3, 10, 25, 50, and 100 times.

kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
proc = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel, iterations = 50)

[gabrielburnworth]

I've added morphological transformation iterations as a function keyword argument.

Iterating does a great job of combining small detached pieces with a main plant. However large internal voids still remain, and increasing iterations tends to increase connections between separate plants, as you've mentioned.

It still remains that the parameters need to be optimized for specific situations.

For comparison of increased kernel size versus increased iterations:

morph kernel size = 15, iterations = 1

morph kernel size = 3, iterations = 10

[gabrielburnworth]

I found a solution to the problem of internal plant voids getting marked as separate plants. The contour search considers contour hierarchy. I changed the contour search from:

contours, hierarchy = cv2.findContours(proc, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

to:

contours, hierarchy = cv2.findContours(proc, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

to return only external contours and ignore internal contours. (Details here)

This:

becomes this:

[roryaronson]

Ah, nice! :+1:

In my testing, the kernel size and iteration choices make a big difference based on how large the plants are in pixels. For example, parameters that worked well for my bird's eye photograph did not work well on your sample images.

In the use case of FarmBot, photos might be taken at different distances from the soil and at different resolutions depending on the camera used and where it is mounted (z-axis vs cross-slide vs overhead the bed/bot). This means plants could commonly be just a handful of pixels wide up to several hundred.

Probably the most useful system to create then is a flexible one, like you've done - where the user can easily tweak the parameters (kernel size, morph type, iterations) to work best for their circumstances. If photos were processed on-the-fly and streamed to a web app widget, then the user could use some sliders to adjust the parameters for real-time feedback and quick tuning of the system. Something like what's been done in this example.

[gabrielburnworth]

I created a GUI with slider bars for blur kernel size, morph kernel size, and iterations:

Calling the function with keyword arguments is still the best way to input other parameters and complicated morphological transformation operations, like:

detect_plants('soil_image.jpg', 
                      blur=15, 
                      array=[[5, 'cross', 'erode', 2],
                             [3, 'rect', 'dilate', 8],
                             [6, 'ellipse', 'close', 1]], 
                      clump_buster=True, debug=True)