Daniel Galvez comments

Results 85 comments of


                                            Daniel Galvez

Training speed is not improved by using a better GPU

A quick way to get a sense of what part is slow is to use the "nvtx" pip's package ability to automatically create an nvtx range for every single python...

CUDAGraph breaks with error msg: "Inplace update to inference tensor outside InferenceMode" for torch.compile with custom backend

I'm not 100% certain about this, but I think the reason is that you are creating the following tensors when inference mode is on: https://github.com/pytorch/pytorch/blob/f3fa560dec727380b3e9c074efe05f0ce715a5ca/aten/src/ATen/cuda/CUDAGeneratorImpl.cpp#L141-L142 This is because you are...

CUDAGraph breaks with error msg: "Inplace update to inference tensor outside InferenceMode" for torch.compile with custom backend

If you want to try my suggestions @wbigat , please goo ahead. It may take me some time to get around to fixing this.

CUDAGraph breaks with error msg: "Inplace update to inference tensor outside InferenceMode" for torch.compile with custom backend

Hi @wbigat I personally don't feel comfortable getting a solution for this immediately, since I'm unfortunately not familiar with how torch.compile interacts with context managers like inference mode and have...

Add safetensor option when saving and restoring models

Reopened. It got stale over the holidays.