torch.Generator, in particular set_state

Open t-vi opened this issue 1 year ago • 2 comments

This comes up in NeMo / NeVA:

https://github.com/NVIDIA/NeMo/blob/32503fd946cedc41152152837c01f95ae4bc6dc6/nemo/collections/nlp/modules/common/megatron/attention.py#L973-L973

cc @tfogal

Oct 01 '24 19:10 t-vi

Hint from the expert (thank you @tfogal): This can be avoided by using flash-attention.

Oct 02 '24 09:10 t-vi

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Apr 16 '25 06:04 stale[bot]