sparktorch icon indicating copy to clipboard operation
sparktorch copied to clipboard

Why is nobody using this?

Open adsk2050 opened this issue 2 years ago • 1 comments

I am new to doing distributed training and inference of ML models. If I have say 4 GPU nodes in a spark cluster, will this library help me train and do inference on the models without having to go into too much detail? If yes, I am wondering why nobody is using this project. If no, what exactly is the function of this library?

adsk2050 avatar Jan 11 '22 07:01 adsk2050

It does have very limited publicity, but based on pypi activity people are using the library. Since the modeling is focused around spark ml/dataframes, if your objective fits within that ecosystem, it could be a fit.

dmmiller612 avatar Jan 11 '22 14:01 dmmiller612