The performance obtained is not ideal

[zxzxzxZZZ]

Hello, thank you for your excellent work. Can you provide a detailed environment configuration for running your code? The results I achieved locally differ significantly from the expectations you provided.

[GiscardBiamby]

Did you try assigning any of the models or inputs to a particular CUDA device, or did you set CUDA_VISIBLE_DEVICES environment var? I don't know why but when I did that I got results like yours. When I change the demo notebook back to original and don't change any CUDA device settings I get results that match the expected result.

[zxzxzxZZZ]

Thank you for your reply, but I haven't made any modifications.

[yan-xu-helm]

Are you seeing message like CUDA OOM --> CPU? I think when you run on a machine without enough vRAM, the model will be moved to CPU and weird things happen. I would suggest running on A100 instance and see if you can get the expected results