bgreenwell
Error: Index larger than maximal for convex hull plots
I have an object (ans1X) trained in xgboost. The dataset has 82,527 rows. I am able to use partial to make a partial dependency plot with one variable. However, when I try and make a convex hull plot (with two variables) using the below code:
p1=partial(ans1X, pred.var = c("fix_eff7", "fix_eff2"), plot = TRUE, chull = TRUE,train=train)
I get the following error:
Error in intI(i, n = x@Dim[1], dn[[1]], give.dn = FALSE) : index larger than maximal 82527
Could you please help out?
It's hard to say without a reproducible example. To narrow down the issue, can you make sure that train is a data frame in the argument train = train?
On second thought, are you using a sparse matrix from the Matrix package?
Yes, I am using a sparse matrix from Matrix package. If I instead use a data frame for `train=train', with the ans1X xgBoost object being trained with the sparse matrix, then I get the following error:
Error in names(pd.df) <- c(pred.var, "yhat") : 'names' attribute [3] must be the same length as the vector [2]
It should work with any combination of training: https://github.com/bgreenwell/pdp/blob/master/slowtests/slowtests-xgboost.R. I'll see if I can find where the true issue is. The error you are getting is thrown from the Matrix package. It would be easier if you could provide a reproducible example? Maybe sample the data if it's not sensitive
?
The sparse matrix works with partial when there is only variable. The problem occurs when there are two variables and I am using chull=TRUE.
I now created the ans1X xgboost object using a dataframe, which was a much slower process (but continued to use the sparse matrix format for cross validation in the previous step), and in partial I used train=train as a dataframe, and now I am getting the convex hull plot. So I found a temporary solution to the problem. Thanks for your help! As the data is sensitive, I am unable to share a reproducible example using that.
As long as you have a reasonable workaround for now! I'll dig into this issue when I find some free time soon and see if I can't figure out the exact issue (might just be a subsetting issue). Will get back to you soon!