ONNX example is not a really full ONNX example

[VladMVLX]

As I notice in example we use exported onnx model only to do prediction, but generation of embeddings still done using pytorch and python. Which is very limited approach if for example I want to deliver to the system that uses onnx runtime only. Is there a way to export onnx for calculating embeddings as well?

[HighPoint]

Is it possible to export the embeddings model into ONNX?

[GeorgeXiaojie]

I have the same question. I want to export all models ImageEncoderViT/PromptEncoder/MaskDecoder, but now only PromptEncoder/MaskDecoder is exported.

[Moeflon]

I was searching for the same thing and came across this Hugging Face repo containing full exports for ViT-B by @visheratin. It includes a link to a Colab that seems to run fine!

Export code: https://github.com/visheratin/segment-anything/commit/13f0fb0152e19c30ae61c5c0394910f5fe1a44c8

https://github.com/facebookresearch/segment-anything/pull/29

[mpottinger]

I was searching for the same thing and came across this Hugging Face repo containing full exports for ViT-B by @visheratin. It includes a link to a Colab that seems to run fine!

Export code: visheratin@13f0fb0

#29

Good news. I am sure people will be working to speed this up. It is perfect, but to be useful I need real-time, even 1 FPS at least

[qjr1997]

why I got '(1, 4, 1200, 1800)' after I run 'masks.shape' instead of (1, 1, 1200, 1800)

[visheratin]

@qjr1997 likely you used the decoder version that returns multiple masks.

[OroChippw]

why I got '(1, 4, 1200, 1800)' after I run 'masks.shape' instead of (1, 1, 1200, 1800)

Because when you convert torch model to onnx model ,No enabled options “return-single-mask” in the file export_onnx_model.py ，If enable the option , the exported ONNX model will only return the best mask[(1, 1, 1200, 1800)], instead of returning multiple masks[(1, 4, 1200, 1800) , 4 masks]. For high resolution images Sry my English is not good ,hope it help you solve the problem😶

[AndreyGermanov]

Until now, I could not find a clear answer to this question and finally had to dive to source code and write the solution myself. Now I can export both image encoder and mask decoder and run inference on ONNX completely. Here is a post about my experience: https://dev.to/andreygermanov/export-segment-anything-neural-network-to-onnx-the-missing-parts-43c8.

[Raspberry-beans]

Until now, I could not find a clear answer to this question and finally had to dive to source code and write the solution myself. Now I can export both image encoder and mask decoder and run inference on ONNX completely. Here is a post about my experience: https://dev.to/andreygermanov/export-segment-anything-neural-network-to-onnx-the-missing-parts-43c8.

Thanks for tutorial. It helped me a lot.

Have you also tried to export SAMAutomaticMask Generator i.e. segment everything mode to ONNX? I want to use segment everything mode, not the points or bboxes.

Thanks!

[AndreyGermanov]

Until now, I could not find a clear answer to this question and finally had to dive to source code and write the solution myself. Now I can export both image encoder and mask decoder and run inference on ONNX completely. Here is a post about my experience: https://dev.to/andreygermanov/export-segment-anything-neural-network-to-onnx-the-missing-parts-43c8.

Thanks for tutorial. It helped me a lot.

Have you also tried to export SAMAutomaticMask Generator i.e. segment everything mode to ONNX? I want to use segment everything mode, not the points or bboxes.

Thanks!

Hello,

The SamAutomaticMaskGenerator is not a different model, that possible to export to ONNX. This class just uses the same SegmentAnything model several times, by applying it to different points of an image in a loop (for example every 50 or 100 points) and then collecting received masks.

You can create this loop on your own, using the SamAutomaticMaskGenerator class source code as a reference: https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py.

[Raspberry-beans]

Thanks for your suggestions.

Can you specify the RAM required for exporting Vit_h to ONNX. I was using Colab with system RAM of 12 GB and program collapsed as it was requiring more than 12 GB. Tried with Vit_b as well but same result.