Thorsten Glüsenkamp

@MarkusSpanring did you solve the issue? Was it related to the driver or the pytorch version 1.13? I have the same issue (https://github.com/ray-project/ray/issues/31451) and use pytorch 1.12.1 currently (I cannot...

I am also working with `ray tune` and use `pytorch-lightning` inside the trainable and see the issue, but not specifically `ray-lightning`. But I understand that with the newest drivers you...

The driver update to 515.xxx does **not** solve the issue for me.

@btravouillon So what happens is that the processes are not properly released in a multiprocess setting (num_workers>0) of a Dataloader in connection to pytorch-lightning and ray (see issue https://github.com/ray-project/ray/issues/31451). I...

This typically comes about in Gaussianization flows when gradients are too large and one ends up in parameter space in regions where certain parametrizations do not work properly anymore. A...

Sorry, I meant *init_params*, instead of *init*. Is now corrected in the other comment.

@Baran-phys Maybe try the following: 1) *IMPORTANT* Make sure you use double precision (setting both the model to double via pdf.double(), and then feed the input to the model as...

@rynewang and that happens since 2.7.0 ? Remember the issue is not present in 2.6.3 and before. If I switch off the custom temp dir, and ray used the default...