Parallelization Within Tasks

[davidcereal]

Hello,

Is there is any local Joblib parallelization possible within parallel_backend. Consider the following slightly modified example from the README. The only thing changed is the SVC estimator is instantiated with n_jobs=2.

register_spark() # register spark backend

clf = svm.SVC(kernel='linear', C=1, n_jobs=2)

with parallel_backend('spark', n_jobs=3):
  scores = cross_val_score(clf, iris.data, iris.target, cv=5)

If there are 2 cores per task (spark.task.cpus=2), would the SVC estimation be parallelized across 2 cores within each of the 3 tasks? If not, is there any way to achieve this?

Thanks in advance!