MobileSAM icon indicating copy to clipboard operation
MobileSAM copied to clipboard

Published 20 hours ago • verified publisher icon ChaoningZhang

Getting mobileSAM into a format that runs on mobile (tflite, torchscript)

Open cummins-orgs opened this issue 11 months ago • 8 comments

Hi mobileSAM creators!

Thanks for the project, it's really awesome.

Maybe this is my stupidity, but I thought I should highlight this: I've essentially tried and failed over a couple of days to get mobileSAM into either torchscript or tflite. Possible that direct conversion to e.g. CoreML would be painless, haven't tried that just yet.

A lot of this isn't traceable to you: converting SAM for use on mobile appears to be non-trivial.

But: one suggestion for the future, if you can spare the time: offer links/walkthrough for getting this project in shape for use in mobile apps.

Or, if that exists, I'd love to follow it..!

Thanks again,

Matt

cummins-orgs avatar Mar 06 '24 22:03 cummins-orgs

Did you make any progress, maybe we can cooperate.

SkyloveQiu avatar Mar 27 '24 23:03 SkyloveQiu

Hi @cummins-orgs @SkyloveQiu ! If you had a look at EdgeSAM, they implemented a CoreML output. I implemented an integration (closed source for now) of it. The thing is that their license is a lot less permissive.

I'm interested by the CoreML output of MobileSAM as well, but did not investigate yet. Don't hesitate to reach out if you're still interested / already made any progress ;)

PierreMardon avatar Jun 27 '24 15:06 PierreMardon

Hi both,

After a lot of trouble the project I was working on got cancelled. Maybe coreml was the way to go! The issues I had in swift were either to do with the actual model file or to do with pre-processing fwiw. Former is much more complex/verbose in swift than python

On Thu, Jun 27, 2024 at 8:54 AM, Pierre Mardon @.***(mailto:On Thu, Jun 27, 2024 at 8:54 AM, Pierre Mardon < wrote:

Hi @.(https://github.com/cummins-orgs) @.(https://github.com/SkyloveQiu) ! If you had a look at EdgeSAM, they implemented a CoreML output. I implemented an integration (closed source for now) of it. The thing is that their license is a lot less permissive.

I'm interested by the CoreML output of MobileSAM as well, but did not investigate yet. Don't hesitate to reach out if you're still interested / already made any progress ;)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

cummins-orgs avatar Jun 27 '24 16:06 cummins-orgs

Ok thanks @cummins-orgs ! I think indeed CoreML is the preferred / simplest path ;)

PierreMardon avatar Jun 27 '24 16:06 PierreMardon

我已经实现了移动端的部署，并且在我的软件中已经应用一年多了

kestraltorah avatar Aug 20 '24 12:08 kestraltorah

Hi @kestraltorah ! Could you provide more insight on how you performed your integration please?

PierreMardon avatar Aug 20 '24 16:08 PierreMardon

使用onnx，编码模型+解码模型

---- Replied Message ---- | From | Pierre @.> | | Date | 08/21/2024 00:11 | | To | ChaoningZhang/MobileSAM @.> | | Cc | kestraltorah @.>, Mention @.> | | Subject | Re: [ChaoningZhang/MobileSAM] Getting mobileSAM into a format that runs on mobile (tflite, torchscript) (Issue #141) |

Hi @kestraltorah ! Could you provide more insight on how you performed your integration please?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

kestraltorah avatar Aug 20 '24 23:08 kestraltorah

Oh right, I didn't even consider it was possible to just use ONNX directly in iOS 🤷‍♂️ I guess I'll try that, thx !

PierreMardon avatar Aug 21 '24 06:08 PierreMardon