michelodu

I am seeing PYTORCH_CUDA_ALLOC_CONF memory errors. I'm trying this.... export 'PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:32' It failed at export 'PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128' which I obtained from https://github.com/CompVis/stable-diffusion/issues/39 . A second thread mentioned a value of 32...

Is it possible to use gradient accumulation to deal with this? Can I invoke train_mono with extra parameters like --gradient_accumulation_steps and expect it to address my memory error?