Investigate the discrepancy in default hyperparams compared to LightGBM

[ogrisel]

Possible culprits:

• shrinkage / learning_rate
• min_samples_leaf
• min_child_samples

See details in https://github.com/ogrisel/pygbm/issues/30#issuecomment-435091127.

[ogrisel]

As @NicolasHug noted, our min_samples_leaf in pygbm is not correct. I would rather implement what LightGBM does, that is reject splits that would result in one of the child nodes having less than min_samples_leaf.

[NicolasHug]

You mean sklearn?

LightGBM is doing something very weird with min_sample_leaf, it looks like it is ignored because of num_leaves (see https://github.com/ogrisel/pygbm/issues/30#issuecomment-435138526)

[guolinke]

@NicolasHug I think you used the wrong parameter name in that code.

[ogrisel]

Indeed. It's actually the pygbm handling of min_samples_leaf that is broken. See: #34.