lightning-hydra-template
lightning-hydra-template copied to clipboard
How to skip the batch when OOM happens?
Hi developers:
Hydra-lightning is a really cool tool and I like it! However, my batch includes highly different size of graphs and sometimes it causes OOM issues. Previously I would manually skip this batch but in hydra-lightning it seems hard to do this. Is it possible to add it in a future version, or how can I skip batch when such batch OOM (out of memory in GPU)?
Xinheng