Matthew Conflitti

batch generation

1

Curious as to why you didn't use a generator for batch creation and then also capped the number of iterations per epoch to 10. I updated the code to this:...

I am properly mounting each worker to the same folder location, but I cannot perform a `spark.read.parquet` without an error. I would guess this is due to the fact it...

Error: One of the request inputs is not valid. on mv or cp function

4

**What happened**: Error occurred on move. Checked the data lake and the files did copy to new location successfully but were not removed from previous location. ``` --------------------------------------------------------------------------- HttpResponseError Traceback...

Beware feature importances

1

https://explained.ai/rf-importance/ Are you aware of this method and its benefits? https://scikit-learn.org/stable/modules/generated/sklearn.inspection.permutation_importance.html?highlight=permutation#sklearn.inspection.permutation_importance Might be a slightly situation using gradient boosted trees, but still an interesting point to address if you could....