AITemplate
AITemplate copied to clipboard
Workaround for SD alt compilation/demo for T4 SM75
Description
Currently compile_alt.py
generates bad images on T4 GPU (SM75).
Related issue - https://github.com/facebookincubator/AITemplate/issues/781
Notebook to reproduce the issue with bad images on T4 GPU - AIT_alt_bad_image.ipynb
I found that the issue with bad images can be fixed if we use the following workaround:
- compile_clip_alt.py - batch higher dim should be 8+
- compile_unet_alt.py - Both lower and upper batch dims should be doubled
I understand that the workaround above uses magic numbers. But it works...
Testing
Tested on T4 and A100 GPUs .
Compiled with different batch ranges (1-8, 2-4, 1-9) and run demo_alt with different batch sizes (1,2,4,8,9) - all images look ok.