PSEUDOTENSOR / Jonathan McKinney
PSEUDOTENSOR / Jonathan McKinney
Here are the parameters: ``` OrderedDict([('bootstrap_type', 'Bayesian'), ('n_estimators', 4000), ('learning_rate', 0.25), ('early_stopping_rounds', 20), ('max_depth', 8), ('one_hot_max_size', 255), ('task_type', 'GPU'), ('eval_metric', 'AUC'), ('objective', 'Logloss'), ('cat_features', [901, 903, 904, 905, 906, 907,...
I'd love to, but it doesn't happen with small data, only with big data and it took several hours to hit that problem.
Please provide some context for this failure so I have some way to know *how* to repro. Thanks!
right, but why can it happen? Hard to generate repro if i have nothing except it happens for gpu training.
I didn't create a re-usable artifact when it happened, must have been like 1/100 chance. I haven't hit lately, so not sure how to reproduce. The other issues I mentioned...
This still happens.
I hit this randomly for no reason with categorical_features as explicitly empty. Has nothing to do with that. The test that hit this normally has passed 1000 times before. ```...
Here's another example. We hit this randomly in testing with about 2 tests showing out of thousands. Again, same thing, very small data, not busy system, plenty of resources, and...
Avoiding Exception "Check failed: (best_split_info.right_count) > (0) at ..." with a regression task
Hi @guolinke I hit the same problem: ``` File "/opt/h2oai/dai/cuda-10.0/lib/python3.6/site-packages/lightgbm_gpu/sklearn.py", line 794, in fit categorical_feature=categorical_feature, callbacks=callbacks, init_model=init_model) File "/opt/h2oai/dai/cuda-10.0/lib/python3.6/site-packages/lightgbm_gpu/sklearn.py", line 637, in fit callbacks=callbacks, init_model=init_model) File "/opt/h2oai/dai/cuda-10.0/lib/python3.6/site-packages/lightgbm_gpu/engine.py", line 251, in...
Avoiding Exception "Check failed: (best_split_info.right_count) > (0) at ..." with a regression task
Building from source like: ``` rm -rf build ; mkdir -p build ; cd build && \ cmake $(GPU_FLAG) $(CUDA_FLAG) -DCMAKE_INSTALL_PREFIX=$$PYTHONPREFIX -DBoost_NO_BOOST_CMAKE=TRUE -DBoost_NO_SYSTEM_PATHS=TRUE -DBOOST_ROOT:PATHNAME=$$BOOSTPREFIX -DBoost_LIBRARY_DIRS:FILEPATH=$BOOSTPREFIX/lib -DOpenCL_LIBRARY=$$CUDA_HOME/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=$$CUDA_HOME/include/ -DBoost_USE_STATIC_LIBS=ON .. &&...