M4.rda problem

[robjhyndman]

@pmontman Yesterday's update is giving me the following installation problem.

> devtools::install_github("carlanetto/M4comp2018")
Downloading GitHub repo carlanetto/M4comp2018@master
✔  checking for file ‘/tmp/RtmpwEPu9C/remotes4b502f4ae37f/carlanetto-M4comp2018-3c75dcd/DESCRIPTION’ ...
─  preparing ‘M4comp2018’:
✔  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  looking to see if a ‘data/datalist’ file should be added
─  building ‘M4comp2018_0.2.0.tar.gz’
   
Installing package into ‘/home/hyndman/R/x86_64-pc-linux-gnu-library/3.6’
(as ‘lib’ is unspecified)
* installing *source* package ‘M4comp2018’ ...
** using staged installation
** R
** data
*** moving datasets to lazyload DB
Warning: file ‘M4.rda’ has magic number 'versi'
  Use of save versions prior to 2 is deprecated
Error in load(zfile, envir = tmp_env) : 
  bad restore file magic number (file may be corrupted) -- no data loaded
ERROR: lazydata failed for package ‘M4comp2018’
* removing ‘/home/hyndman/R/x86_64-pc-linux-gnu-library/3.6/M4comp2018’
* restoring previous ‘/home/hyndman/R/x86_64-pc-linux-gnu-library/3.6/M4comp2018’
Error in i.p(...) : 
  (converted from warning) installation of package ‘/tmp/RtmpwEPu9C/file4b5063ce3543/M4comp2018_0.2.0.tar.gz’ had non-zero exit status

[config-i1]

I have the same issue.

[camdenhu]

I encountered the same error. Is there any plan to fix it?

[andybega]

A temporary solution is to install one of the previous commits. This works for me:

remotes::install_github("carlanetto/M4comp2018", ref = "6e75d59eb30e47cbb6bd5093d5bf2515493a6050")

The commit is from July 2018. 6e75d59eb30e47cbb6bd5093d5bf2515493a6050

[camdenhu]

Thank you, that worked for me.

[pmontman]

Thank you for reporting the issue. The package includes the top 25 ranked submissions to the competition, their point and interval forecasts. The size of the dataset is now too big for install_github so we are now using a github relase for distributing the package. Instructions on installation and exploring the new data structures are in the projects' README.

We will try to get a cleaner solution and include @andybega solution for those not interested in the submission info.

[tm1611]

I encountered a similar issue following the instructions on installation in the projects' README. Is there either

1. A plan to fix it or
2. another option to access the results of the top ranked methods in R?

Note that a previous version of M4comp2018 was already installed on my system.

Command

install.packages("https://github.com/carlanetto/M4comp2018/releases/download/0.2.0/M4comp2018_0.2.0.tar.gz", repos=NULL)

Error message

Error : cannot allocate vector of size 1003.4 Mb
ERROR: lazydata failed for package 'M4comp2018'
* removing 'C:/Users/TM/Documents/R/win-library/3.5/M4comp2018'
* restoring previous 'C:/Users/TM/Documents/R/win-library/3.5/M4comp2018'
In R CMD INSTALL
Warning in install.packages :
  installation of package ‘[...]/downloaded_packages/M4comp2018_0.2.0.tar.gz’ had non-zero exit status

System

• Previous version of M4comp2018 already installed
• Platform: x86_64-w64-mingw32 (Windows)
• R version 3.6.1 (2019-07-05)

I tested this on my own and one additional platform. In addition, I tested this on a R-kernel run on an AWS SageMaker instance (ml.t2.medium). Moreover, I tried other suggested fixes (i.e. using garbage control gc(), clear workspace, memory and eliminated all other running processes).

Thank you.

[robjhyndman]

@pmontman. One way to deal with this is to not put the large data files in the package. Instead, you can include a function get_m4data() which downloads the data from the git repository. You just need to put the associated rda file in some folder that is listed in .Rbuildignore.

[pmontman]

Thank you all for your reports and suggestions, I have checked the command to install from the github release and it is working for me, in three different R installations. Before we move into other workarounds, @tm1611, Could you try to manually download the release: https://github.com/carlanetto/M4comp2018/releases/download/0.2.0/M4comp2018_0.2.0.tar.gz and check to manually load the file data/M4.rda of the package? This can be done by using the readRDS() function in R. M4.rda is the file that contains the time series, true future values and submissions for the M4 competition. This could serve you as a temporary workaround and help me identify the problem, since your reported error: Error : cannot allocate vector of size 1003.4 Mb Might not be a problem with the installation but with the size of the dataset.

[tm1611]

Thank you all for your reports and suggestions, I have checked the command to install from the github release and it is working for me, in three different R installations. Before we move into other workarounds, @tm1611, Could you try to manually download the release: https://github.com/carlanetto/M4comp2018/releases/download/0.2.0/M4comp2018_0.2.0.tar.gz and check to manually load the file data/M4.rda of the package? This can be done by using the readRDS() function in R. M4.rda is the file that contains the time series, true future values and submissions for the M4 competition. This could serve you as a temporary workaround and help me identify the problem, since your reported error: Error : cannot allocate vector of size 1003.4 Mb Might not be a problem with the installation but with the size of the dataset.

Manually downloading the package and importing only the M4.rda in R worked fine. Thank you very much.

[ssh352]

A temporary solution is to install one of the previous commits. This works for me:

remotes::install_github("carlanetto/M4comp2018", ref = "6e75d59eb30e47cbb6bd5093d5bf2515493a6050")

The commit is from July 2018. 6e75d59

same issue and this worked for me.