Onnx export

[gianmarcocr]

Can you upload a script to correctly export the entire model as onnx ?

[lkeab]

Thanks. We will support it in the future. But it should be a similar process to here.

[juanilarregui]

First of all, thanks for open-sourcing your work.

I tried running the original SAM Jupyter notebook for exporting the decoder in ONNX format.

I was able to correctly run the cells where the model is laoded, using SAM-HQ vit-h checkpoint. However, when running the cell that effectively exports the model as ONNX, I'm getting:

`[...]

File ~/Downloads/sam-hq/segment_anything/utils/onnx.py:120, in SamOnnxModel.forward(self, image_embeddings, point_coords, point_labels, mask_input, has_mask_input, orig_im_size) 117 sparse_embedding = self._embed_points(point_coords, point_labels) 118 dense_embedding = self._embed_masks(mask_input, has_mask_input) --> 120 masks, scores = self.model.mask_decoder.predict_masks( 121 image_embeddings=image_embeddings, 122 image_pe=self.model.prompt_encoder.get_dense_pe(), 123 sparse_prompt_embeddings=sparse_embedding, 124 dense_prompt_embeddings=dense_embedding, 125 ) 127 if self.use_stability_score: 128 scores = calculate_stability_score( 129 masks, self.model.mask_threshold, self.stability_score_offset 130 )

TypeError: MaskDecoderHQ.predict_masks() missing 1 required positional argument: 'hq_features'`

Any help is much appreciated.

[gianmarcocr]

i'm facing the same problem as @juanilarregui . i'm trying to fix it, if I manage to do so I will copy here the solution

[unriddler]

same here.

[juanilarregui]

Hi @lkeab , any advice you could provide for that specific error?

[lkeab]

"TypeError: MaskDecoderHQ.predict_masks() missing 1 required positional argument: 'hq_features'`". By check this error log, I would suggest you to modify the here and providing the hq-features required by HQ-sam.

[juanilarregui]

I'm not being able to get hq_features (that depends on interm_embeddings for example) inside that class, given that interm_embeddings for what I'm seeing is an instance attribute of SamPredictor. Am I correct?

[thlz998]

same here.

[lkeab]

hi, we are working on it and will support it soon.

[David-19940718]

hi, we are working on it and will support it soon.

Hi, is there any approximate date available?

[ymq2017]

Hi, we have updated the script and example colab notebook for exporting and using ONNX model.

[David-19940718]

Hi, we have updated the script and example colab notebook for exporting and using ONNX model.

Thank you for your nice work! But, how can we separate the encoder and decoder into separate onnx files (like traditional seq2seq models) for flexible deploy? Otherwise, if we export the model with an encoder and decoder combined, that will have to run the encoder on every prompt, which is a huge overhead.

[ymq2017]

Hi, we have updated the script and example colab notebook for exporting and using ONNX model.

Thank you for your nice work! But, how can we separate the encoder and decoder into separate onnx files (like traditional seq2seq models) for flexible deploy? Otherwise, if we export the model with an encoder and decoder combined, that will have to run the encoder on every prompt, which is a huge overhead.

Hi, this pull request may help you.