facebookresearch
Automatic mask generator onnx
How to convert Sam automatic mask generator (generate ) function to onnx to gain speed up?
Hi @maheshs11 did you find a way to segment the whole image withh onnx model please
Did you find a solution? @maheshs11 @assia855 @praveenporsche @JinghuiZhou
Did you find a solution? @maheshs11 @assia855 @praveenporsche @JinghuiZhou
can not find a solution right now
For anyone still wanting to do this, the auto-mask procedure involves a lot of dynamic memory use, so it would be difficult/impossible to do entirely in onnx. Instead, it would be easier to use the SAM onnx model for generating mask predictions and then handle the auto-mask filtering logic outside of onnx (in python or C++ or whatever is executing the onnx model). The code that needs to be implemented is mostly inside the _process_batch function, which filters out masks based on various threshold settings, along with the final non-max suppression step which removes duplicate masks.
If you are trying to implement this, I have a slightly simplified version of the filtering logic and final NMS step, which might be easier to follow and includes a visualization (though it doesn't include the cropping logic from the original SAM implementation).
