need xformers whl for torch 2.0.1+cu117 for Unity Project that does realtime depth estimation

[brentjohnston]

❓ Questions and Help

I am working on a Unity project that seems pretty heavily integrated with cuda 11.7, and I don't know how hard it would be to upgrade to 11.8. This project converts 2D video and images to 3D using depth estimation models like the new tiktok depth-anything model, beit, etc.

The problem is with the depth-anything model it runs at like 2 fps when it does the realtime conversion. I had an idea to install xformers, but I get an error that xformers is not compatible with torch 2.01 and cuda 11.7. Is there any wheel out there that could make this work, and do you think this would speed up the realtime 2d to 3d processing the model does with xformers? I have looked everywhere I cannot find a whl. Thanks.

[NeedsMoar]

11.8 should be compatible with 11.7. There's no 11.7 version; I've built xformers and flash-attn-2 with 12.3 while running torch built with the newest tools they use (12.1) and it works fine, so just pull a 2.0.1+11.8 wheel. It wont make 2 fps realtime though...

[brentjohnston]

I take it back I checked and it runs at about 4 fps on my 4090 in depthviewer with depth-anythibg large model.

If I try your idea with the xformers and that whl, do you think it would give any sort if bump in framerate for depth-anything though or is the performce bumps more for AI image generation apps? Even just a few fps would be great.