Force GPU memory limit

[QingtaoLi1]

As mentioned in this question, PyTorch now supports limiting GPU memory usage, which helps better management and planning on it.

[NiklasGustafsson]

I'm searching for the C++ code that implements it. Haven't found it yet.

[dayo05]

@NiklasGustafsson I found method implementation of set_per_process_memory_fraction is exists on torch/csrc/cuda/Module.cpp::_cuda_setMemoryFraction I don't know about how pytorch implementing native method to python, but this might be help :D

[dayo05]

Maybe this method is doing that: c10::cuda::CUDACachingAllocator::setMemoryFraction(fraction, device);

[NiklasGustafsson]

CUDACachingAllocator

Which header file is that declared in? I don't find it in torch/cuda.h

[dayo05]

CUDACachingAllocator

Which header file is that declared in? I don't find it in torch/cuda.h

c10/cuda/CUDACachingAllocator.cpp here in 927 line

c10/cuda/CUDACachingAllocator.h this is header

[dayo05]

@NiklasGustafsson I'm trying to implement this method but, this method is only available on CUDA-backend api. Is there any place to locate cuda-only-apis?

[NiklasGustafsson]

No, because the interop layer is backend-independent, so it doesn't link to anything that isn't available in both.