Using customized subset of features when growing tree for different depths

[raff7]

Hi,

I came across issue #3923 from 2018 while exploring ways to apply feature constraints based on tree depth. It appears this idea was discussed but never implemented.

To address this, I’ve implemented a simple version using a new parameter, depth_feature_sets, which defines allowed features at each depth as a list of lists. For example:

depth_feature_sets = [[1, 2, 3], [1, 2], [2, 3]] This means:

• Depth 1: use features [f1, f2, f3]
• Depth 2: use [f1, f2]
• Depth 3: use [f2, f3]
• Depth > 3: unconstrained

The implementation is minimal and modeled after feature_interaction_constraints.

This feature is valuable when certain features dominate the model, suppressing others. In my case, while implementing a product recommendation system that would predict purchase propensity for any customer-product pair, user features overwhelmed product features. By constraining the final tree levels to only use product features, I restored model variability across products.

Would you be open to including this in the repo? I’m happy to clean up the implementation and open a PR.

Thanks!

[trivialfis]

Could you please help test the feature_weights as well, and see if it can achieve similar results?

[raff7]

Is that an already implemented feature that just needs testing? Is there some known issue with it or is it just a double check you are asking me?

I can check it out in case

[trivialfis]

It's already implemented and tested. I'm curious if you have tried it and whether it can help your use case.

[raff7]

ah yea that's a good idea, thank you ill give it a try

[trivialfis]

Hi @raff7 , any update?

[trivialfis]

Feel free to reopen if there's further information.