Albert Zeyer

> Can I hook the tensor `__del__`? One solution to this ([via](https://github.com/albertz/playground/blob/master/torch-tensor-track.py)): ```python class _TensorHandle: def __init__(self, tensor): self.tensor_ref = weakref.ref(tensor) # not really used here... def __del__(self): print("out-of-scope") def...

I wonder a bit about [`torch.utils.checkpoint`](https://pytorch.org/docs/stable/checkpoint.html): Why do I need to specify the inputs for the function? What happens when I forget some? E.g. the example speaks about LSTM and...

I also wonder how [`torch.utils.checkpoint`](https://pytorch.org/docs/stable/checkpoint.html) is supposed to be used in practice. The outputs of the checkpointing function are potentially anyway stored elsewhere for backprop, as those are not under...

> I also asked the [question in the PyTorch discussion forum](https://discuss.pytorch.org/t/gradient-checkpointing/205416). I got some quite useful answers which clarify my questions. I have a very hacky idea: I can somehow...

Btw, I also have an idea on how to write a test case for this: We use some very similar code as in the example. Once without gradient checkpointing: ```python...

Note, one problem with deferring the `saved_tensors_hook.__exit__`: If there are any other (unrelated) `saved_tensors_hook.__enter__` in the meantime, and then we call our `saved_tensors_hook.__exit__` while the new scope is active, this...

See https://github.com/rwth-i6/i6_core/issues/636. I think at least we can require Python >=3.10.

Btw, for the newer Python versions, what are relevant features for RETURNN?