MiDaS icon indicating copy to clipboard operation
MiDaS copied to clipboard

device "mps" for Apple silicon - strange output

Open RaceBo opened this issue 1 year ago • 1 comments

I've tried to run MiDaS on Apple silicon with pytorch nightly with mps (metal performance shader) support.

I've change two lines of code to get this working: 28 - device = torch.device("cuda" if torch.cuda.is_available() else "mps") and 127 - mode="bilinear",

The change to "bilinear" is necessary because bicubic is not supported currently for the M1 native implementation. Influence on output quality is neglectable in my experience (I did a few comparing tests on "CPU" with only this parameter changed)

This way I could speed up inference by a whopping factor of 6. But: Output quality detoriates to not-usable. I've got virtually no experience with python and pytorch - I'm basically a hobbyist creator and dad who wants to have a depth map on selected gopro footage of his son to fake depth of field into it. So i do not know where to look at.

Legacy Midas V2 seems to work okay and delivers the same output on CPU and MPS: 000727v2cpu CPU 000727v2mps MPS

DPT however, output becomes unusable with mps: 000727dptmps MPS

000727dptcpu CPU

Any ideas? Can someone give me a hint?

RaceBo avatar Aug 29 '22 17:08 RaceBo

any updates?

duckheada avatar Aug 03 '23 14:08 duckheada