Error running 2015 paper example

[wilkinsond]

Hello,

I'm currently trying to run the example from your 2015 paper using the code from the tutorial in the appendices and I'm running into a few issues. I've gotten past the problem of the BBS data now being supplied in a different format using code from one of my supervisors @goldingn here plus a few more minor tweaks to column names, but I'm having problems running the model itself.

When running the # Cross-validation section of the mistnet_cross-validation.R script I get the following error in R:

Error in crossEntropyRegGrad(y = y, yhat = yhat, a = a, b = b) : dims [product 6660] do not match the length of object [0]

We are pretty sure the error occurs here as there is nothing to define b, and b returns as NULL when we did some exploration. We have gotten past this by explicitly setting a value for b (b = a, as that also appeared in a formula during exploration). This then allows the # Cross Validation code to start to run.

The next error we receive though occurs ~20 minutes into the run time of the # Cross Validation code causing the code to fail. I throws up this error message:

Error in importanceSamplingEvaluation(net, newdata = scale(env)[in.train, : unused argument (verbose = FALSE)

Do you have any idea what is causing these errors? Or a potential fix?

Regards, David

[goldingn]

Hey Dave,

We still haven't managed to resolve this. I'm guessing the issue is that mistnet (v1) has evolved since you ran the experiments, so the arguments in the BBS evaluation scripts no longer correspond to the version of mistnet at the head of master.

We could hack around in there and make some changes, but the aim is to compare the code at time of publication against other datasets, so we'd rather get the specific version that works if possible.

I just remembered seeing the following line in your preprint (it's not in the MEE version though):

The analyses in this paper had essentially all been run by the commit at https://github.com/davharris/mistnet/tree/1e2eaaeabf9b4b4360f19b00c0d06508578d7f15.

I tried installing directly from that commit in case that's the version to use, with:

devtools::install_github('davharris/mistnet@1e2eaaeabf9b4b4360f19b00c0d06508578d7f15')

but that fails with

Error in .install_package_code_files(".", instdir) : 
files in '/private/var/folders/y8/bwn1d6dx6hs042dp0xz246vc0000gn/T/RtmpyU4qvG/devtools26d249f94bd1/davharris-mistnet-1e2eaae/R' missing from 'Collate' field:
  PCPlot.R
  findLogLik.R
  predict.R
ERROR: unable to collate and parse R files for package ‘mistnet’
* removing ‘/Users/nick/Library/R/3.2/library/mistnet’
Error: Command failed (1)

So then I tried installing release 0.2.0, as that was the first release after that mentioned commit and after most of the scripts in the BBS evaluation directory were written, with:

devtools::install_github('davharris/[email protected]')

which does install. I haven't tried running this version on the data yet (@wilkinsond, can you give this version a go and see if the same errors appear?) but wanted to check with you to see if that's the right one before ploughing on.

[wilkinsond]

Installing mistnet v0.2.0 fixed the issue with the # Cross Validation code, so that now runs smoothly until it hits the # Fit Final Model section. It looks like its failing at this line here since it gives the following error:

Error in which.max(logliks[, 2]) : (list) object cannot be coerced to type 'double'

Since using an older version of mistnet helped I thought I'd try an older version of dplyr as well since that is used in this section but I'm currently having issues getting an older version (v0.2.0) from the same time as v0.2.0 of mistnet to install.

[wilkinsond]

Think I've figured it out.

The which.max() function in net seems to have an issue with logliks being a data.frame (I guess which.max has a reduced list of accepted inputs since the paper was out?). I added the following line of code after this one and it now works.

logliks <- data.matrix(logliks, rownames.force = TRUE)

[davharris]

Hey @wilkinsond, @goldingn, sorry for the delay getting back to you!

Does it look like everything is working now?

If not, I'd be happy to help now that I'm not traveling. If so, would you mind sending me what you have so I can make a fork that won't have these problems?

Thanks!