max_features

[mvalsania]

How should we talk about the impact of max_features on the dvc?

I am trying to reference the formula we have ( O(klog(kd)) ) but I am still conflicted with the fact that we are not technically affecting the number of dimensions with this hyperparameter, we are affecting the number of dimensions that we take into consideration before deciding which dimension would lead to the best split. What is the right way to think about this? Should we think of this hyperparameter as indirectly and artificially affecting d?

Thanks!

[mikeizbicki]

The $d$ in the VC dimension formula for the decision tree is the dimension of the data space and unaffected by the max_features hyperparameter. The later does not affect the VC dimension, and is primarily used for reducing the runtime of training.

[mvalsania]

Thanks!