genrl
genrl copied to clipboard
SforAiDl
Add PyTorch Lightning support
PyTorch Lightning automates away most of the engineering required on PyTorch for features like multi-GPU distributed training, early stopping, etc. More details can be found on the blog for the latest version here: https://medium.com/pytorch/pytorch-multi-gpu-metrics-and-more-in-pytorch-lightning-0-8-1-b7cadd04893e.
Any thoughts on how you're going to go about it? Not sure but I think we can use model parallelism but not data parallelism unless we have offline RL datasets (which we should add support for too)
@sauradefy99 Did you think on this? I think a good first step could be in trying out Offline RL?
Multi-GPU distributed training isn't the same at least in the online/off-policy case, it might be for Offline/Batch RL.
I think this issue needs to be reopened. PLightning support should be critical
Hey @ugurkanates , thanks for your interest in GenRL! We'd closed this initially because a lot of our team is working on some or the other thing right now. Would you be interested in helping us with this? Let us know if you can!
Yes I'm actually looking forward because we are trying to integrate RL frameworks to our codebase and we still want benefits from PL Lightning to apply.
Sounds great! I'll assign you to this then. Let us know how you're gonna go about this.
cc: @Sharad24
Sorry but we instead decided to go with RLLib for now. Maybe later someone(or me) will keep updated on this task