sparktorch
sparktorch copied to clipboard
Why is nobody using this?
I am new to doing distributed training and inference of ML models. If I have say 4 GPU nodes in a spark cluster, will this library help me train and do inference on the models without having to go into too much detail? If yes, I am wondering why nobody is using this project. If no, what exactly is the function of this library?
It does have very limited publicity, but based on pypi activity people are using the library. Since the modeling is focused around spark ml/dataframes, if your objective fits within that ecosystem, it could be a fit.