ExtraTreesQuantileRegressor takes `n_jobs=-1` but doesn't use all (virtual) cores worth of threads to train the model

[graemedatanerds]

The title is pretty self explanatory. My code:

from skgarden.quantile import ExtraTreesQuantileRegressor
...
X_train, X_val, y_train, y_val = load_the_data(...)
model = ExtraTreesQuantileRegressor(n_jobs=-1, criterion='mse', max_depth=None, max_features='auto', max_leaf_nodes=None, n_estimators=10)
model.fit(X_train, y_train)
...

I've tried with various simpler configs and also tried with n_jobs>1, and also tried using RandomForestQuantileRegressor to no avail. I have this bug on multiple computers too, under different hardware too. I looked at the code and it looks like you pass n_jobs properly through to the base scikit learn class instances. Can any insight be offered here?

Great package, BTW.

[mzhao94]

Have there been any updates on this? I haven't been able to get this to work with anything other than the default njobs. If I try n_jobs = -1 for RandomForestQuantileRegressor the code never finishes.

[kyle-pena-nlp]

I am also seeing this behavior as well. Curious if there has been any progress on this issue.

[kyle-pena-nlp]

Being under time constraints, my temporary solution to this problem was as follows:

In .venv\Lib\site-packages\skgarden\quantile\ensemble.py (your path will vary), I added the following constructor to BaseForestQuantileRegressor.

def __init__(self,
             base_estimator,
             n_estimators=100,
             estimator_params=tuple(),
             bootstrap=False,
             oob_score=False,
             n_jobs=None,
             random_state=None,
             verbose=0,
             warm_start=False):
    super(BaseForestQuantileRegressor, self).__init__(
            base_estimator,
            n_estimators=n_estimators,
            estimator_params=estimator_params,
            bootstrap=bootstrap,
            oob_score=oob_score,
            n_jobs=n_jobs,
            random_state=random_state,
            verbose=verbose,
            warm_start=warm_start)`

Then, n_jobs were n_jobbey.

@MechCoder --- do you want to work together to find the right solution?