xgboost
xgboost copied to clipboard
dmlc
Dev/unified dispatching prototype
In continuous of #5659 and #6212. Here I present a way for dispatching the various devices (cpu/cuda device/oneapi device). This request contains only the changes being related to all the devices. The code for oneapi devices support is planned to be added later.
The main idea of dispatching was discussed in #6212. A new global parameter called device_selector is added. This parameter determines the device where the calculations will be made as well as the specific kernel that will be executed. So if the user configures XGBoost by the following parameters:
clf = xgboost.XGBClassifier(... , objective='multi:softmax', tree_method='hist')
the cpu version of the library will be executed. But if the user add device_selector="oneapi:gpu":
clf = xgboost.XGBClassifier(... , device_selector='oneapi:gpu', objective='multi:softmax', tree_method='hist')
the specific code for oneapi GPU will be used.
For cuda the relative logic is not implemented, thus for this case it is just an alternative way for setting the gpu_id. For saving backward compatibility with the existing user code, the priority of gpu_id is made higher.
The additional feature added by this request is an independent specification of devices for fitting and prediction. If the user specifies device_selector='fit:oneapi:gpu; predict:cpu', oneapi GPU will be used for fitting, and CPU will be used for prediction.
Thank you for working on this! I also wrote a higher level RFC https://github.com/dmlc/xgboost/issues/7308 for future device dispatching, which should be complementary to this PR.
I will look into this in more detail later.
This is mostly complete now. https://github.com/dmlc/xgboost/issues/7308#issuecomment-1435248520