Auto import for the predict method

[jknowles]

If you ensemble a series of models using external packages (e.g. mda or glmnet) and then clear your workspace and R session and load the caretEnsemble object and attempt to use the predict method, it fails.

This is because the predict method for each individual model type is not necessarily in the namespace. The predict.caretEnsemble should be able to identify and import the predict methods necessary to generate predictions for each model type to be ensembled.

[zachmayer]

Here's a script that reproduces the error:

library(glmnet)
library(mda)
library(rpart)
x <- as.matrix(iris[,-c(4:5)])
m1 <- glmnet(x, iris[,4])
m2 <- mda(Species~., iris)
m3 <- rpart(Species~., iris)
save(m1, m2, m3, x, file='~/Desktop/m.rda')

In a new session:

load('~/Desktop/m.rda')
p1 <- predict(m1, x)
p2 <- predict(m2, iris)
p3 <- predict(m3, iris)

glmnet and mda fail, rpart works. Honestly, this seems to me to be a bug in the glment and mda packages. Saved S3 objects should know how to find their methods....

[zachmayer]

It seems like the problem is the child classes (elnet and mda) that inherit from parent classes (glmnet, fda). Neither package defines S3 methods for the child classes,

[zachmayer]

I think the problem would be fixed if glmnet included S3method("predict", "elnet") and mda included S3method("predict", "mda") in their namespace.

I can't find an issue tracker for either package, but they're both maintained by Trevor Hastie (A fairly well-known statistician). Do you think it's worth emailing him a feature request?

[zachmayer]

Hmmm, mda includes S3method("predict", "mda") and S3method("predict", "fda") in it's namespace, so that's not the issue.

However, mda does NOT export it's predict function. Maybe adding predict.mda to mda's exports and S3method("predict", "elnet") to glmnet would fix it.

[jknowles]

@zachmayer , we should definitely e-mail him. I think these are very useful prediction functions and I have at least 1 major bug in a production system that I have to write a hacked up workaround as a result of this. Would be very beneficial to others I think!

[zachmayer]

I emailed Trevor Hastie and he said he'll fix the issue if he can find a mechanism for doing so. What other packages trigger this error?

[jknowles]

I'll run a check. I have run into it on a few regression models as well -- I should be able to get a report some time this week.

[zachmayer]

I'll checking with Trevor Hastie sometime early next year and see if he thinks this is fixable.

[jknowles]

I'll try to build a test case for this by looping through caret methods with my datasets.

[zachmayer]

Sounds good!

[zachmayer]

I pinged Prof. Hastie about this, and he said it's on his list, but he's not sure when he'll get to it.

Perhaps some industrious person could submit a patch to him. I'm note sure where the mda and glment source code lives.