pkgr icon indicating copy to clipboard operation
pkgr copied to clipboard

pkgr appears to ignore repo customization on macOS

Open copernican opened this issue 4 years ago • 2 comments

pkgr appears to ignore a repo customization to install source packages. With this pkgr.yml

Version: 1

Packages:
  - dplyr

Repos:
  - MPN: https://mpn.metworx.com/snapshots/stable/2020-09-20

Customizations:
  Repos:
    - MPN:
        Type: source

Lockfile:
    Type: renv

I see

➜  /tmp pkgr plan
INFO[0000] Installation would launch 12 workers
INFO[0000] R Version 4.0.3
INFO[0000] OS Platform x86_64-apple-darwin19.6.0
INFO[0000] found installed packages                      count=20
INFO[0000] Default package installation type:  binary
INFO[0000] 1009:1009 (binary:source) packages available in for MPN from https://mpn.metworx.com/snapshots/stable/2020-09-20
INFO[0000] Package installation cache directory:  /Users/SeanW/Library/Caches/pkgr
INFO[0000] Database cache directory:  /Users/SeanW/Library/Caches/pkgr/r_packagedb_caches
INFO[0000] package installation status                   installed=20 not_from_pkgr=0 outdated=0 total_packages_required=20
INFO[0000] package installation sources                  MPN=20 tarballs=0
INFO[0000] package installation plan                     to_install=0 to_update=0
INFO[0000] Library path to install packages: renv/library/R-4.0/x86_64-apple-darwin19.6.0
INFO[0000] resolution time 23.934694ms

and the resulting pkgr install takes about 2.2 seconds, which isn't nearly enough to install from source. This is pkgr 2.0.1:

➜  /tmp pkgr | grep version
pkgr cli version v2.0.1

copernican avatar Dec 04 '20 00:12 copernican

can you run plan with a loglevel of debug - I'm unable to reproduce - and I see source when i run.

⋊> /t/test pkgr plan --loglevel=debug                                                                                      06:02:24
INFO[0000] Installation would launch 16 workers
INFO[0000] R Version 3.6.3
INFO[0000] OS Platform x86_64-apple-darwin15.6.0
INFO[0000] Package Library will be created               path=renv/library/R-3.6/x86_64-apple-darwin15.6.0
DEBU[0000] packages database - type: source, url: https://mpn.metworx.com/snapshots/stable/2020-09-20/src/contrib/PACKAGES
DEBU[0000] packages database - type: binary, url: https://mpn.metworx.com/snapshots/stable/2020-09-20/bin/macosx/el-capitan/contrib/3.6/PACKAGES
DEBU[0000] potential packages                            num_pkgs=1013 url="https://mpn.metworx.com/snapshots/stable/2020-09-20/src/contrib/PACKAGES"
DEBU[0000] invalid package constraint                    pkg=foreign version="R (>= 4.0.0)"
DEBU[0000] invalid package constraint                    pkg=RODBC version="R (>= 4.0.0)"
DEBU[0000] invalid package constraint                    pkg=XML version="R (>= 4.0.0)"
DEBU[0000] potential packages                            num_pkgs=1009 url="https://mpn.metworx.com/snapshots/stable/2020-09-20/bin/macosx/el-capitan/contrib/3.6/PACKAGES"
INFO[0000] Default package installation type:  binary
INFO[0000] 1009:1009 (binary:source) packages available in for MPN from https://mpn.metworx.com/snapshots/stable/2020-09-20
INFO[0000] Package installation cache directory:  /Users/devinp/Library/Caches/pkgr
INFO[0000] Database cache directory:  /Users/devinp/Library/Caches/pkgr/r_packagedb_caches
DEBU[0000] package repository set                        pkg=dplyr relationship="user package" repo=MPN type=source version=1.0.2
DEBU[0000] package repository set                        pkg=digest relationship=dependency repo=MPN type=source version=0.6.25
DEBU[0000] package repository set                        pkg=fansi relationship=dependency repo=MPN type=source version=0.4.1
DEBU[0000] package repository set                        pkg=generics relationship=dependency repo=MPN type=source version=0.0.2
DEBU[0000] package repository set                        pkg=glue relationship=dependency repo=MPN type=source version=1.4.2
DEBU[0000] package repository set                        pkg=rlang relationship=dependency repo=MPN type=source version=0.4.7
DEBU[0000] package repository set                        pkg=R6 relationship=dependency repo=MPN type=source version=2.4.1
DEBU[0000] package repository set                        pkg=pkgconfig relationship=dependency repo=MPN type=source version=2.0.3
DEBU[0000] package repository set                        pkg=magrittr relationship=dependency repo=MPN type=source version=1.5
DEBU[0000] package repository set                        pkg=utf8 relationship=dependency repo=MPN type=source version=1.1.4
DEBU[0000] package repository set                        pkg=assertthat relationship=dependency repo=MPN type=source version=0.2.1
DEBU[0000] package repository set                        pkg=crayon relationship=dependency repo=MPN type=source version=1.3.4
DEBU[0000] package repository set                        pkg=purrr relationship=dependency repo=MPN type=source version=0.3.4
DEBU[0000] package repository set                        pkg=cli relationship=dependency repo=MPN type=source version=2.0.2
DEBU[0000] package repository set                        pkg=vctrs relationship=dependency repo=MPN type=source version=0.3.4
DEBU[0000] package repository set                        pkg=tidyselect relationship=dependency repo=MPN type=source version=1.1.0
DEBU[0000] package repository set                        pkg=lifecycle relationship=dependency repo=MPN type=source version=0.2.0
DEBU[0000] package repository set                        pkg=ellipsis relationship=dependency repo=MPN type=source version=0.3.1
DEBU[0000] package repository set                        pkg=pillar relationship=dependency repo=MPN type=source version=1.4.6
DEBU[0000] package repository set                        pkg=tibble relationship=dependency repo=MPN type=source version=3.0.3
INFO[0000] package installation status                   installed=0 not_from_pkgr=0 outdated=0 total_packages_required=20
INFO[0000] package installation sources                  MPN=20 tarballs=0
INFO[0000] package installation plan                     to_install=20 to_update=0
INFO[0000] Library path to install packages: renv/library/R-3.6/x86_64-apple-darwin15.6.0
INFO[0000] resolution time 317.573673ms

in addition with debug you'll see the installer information

pkgr install --loglevel=debug
...
DEBU[0002] cmd output                                    exitCode=0 package=pkgconfig stderr="* installing *source* package ‘pkgconfig’ ...\n** package ‘pkgconfig’ successfully unpacked and MD5 sums checked\n** using staged installation\n** R\n** inst\n** tests\n** byte-compile and prepare package for lazy loading\n** help\n*** installing help indices\n** building package indices\n** testing if installed package can be loaded from temporary location\n** testing if installed package can be loaded from final location\n** testing if installed package keeps a record of temporary installation path\n* creating tarball\npackaged installation of ‘pkgconfig’ as ‘pkgconfig_2.0.3.tgz’\n* DONE (pkgconfig)\n" stdout=
DEBU[0002] cmd output                                    exitCode=0 package=assertthat stderr="* installing *source* package ‘assertthat’ ...\n** package ‘assertthat’ successfully unpacked and MD5 sums checked\n** using staged installation\n** R\n** tests\n** byte-compile and prepare package for lazy loading\n** help\n*** installing help indices\n** building package indices\n** testing if installed package can be loaded from temporary location\n** testing if installed package can be loaded from final location\n** testing if installed package keeps a record of temporary installation path\n* creating tarball\npackaged installation of ‘assertthat’ as ‘assertthat_0.2.1.tgz’\n* DONE (assertthat)\n" stdout=
DEBU[0002] cmd output                                    exitCode=0 package=magrittr stderr="* installing *source* package ‘magrittr’ ...\n** package ‘magrittr’ successfully unpacked and MD5 sums checked\n** using staged installation\n** R\n** inst\n** tests\n** byte-compile and prepare package for lazy loading\n** help\n*** installing help indices\n** building package indices\n** installing vignettes\n** testing if installed package can be loaded from temporary location\n** testing if installed package can be loaded from final location\n** testing if installed package keeps a record of temporary installation path\n* creating tarball\npackaged installation of ‘magrittr’ as ‘magrittr_1.5.tgz’\n* DONE (magrittr)\n" stdout=
DEBU[0002] cmd output                                    exitCode=0 package=pkgconfig stderr="* installing *binary* package ‘pkgconfig’ ...\n* DONE (pkgconfig)\n" stdout=
DEBU[0002] package install request completed             WID=4 duration=2.255447994s package=pkgconfig
DEBU[0002] pkgconfig
...

dpastoor avatar Dec 04 '20 11:12 dpastoor

It appears that this may be a caching issue. If I run pkgr clean cache and then repeat the above steps, it takes 42 seconds, and with --loglevel=debug I see the various compilation messages. So, what seems to be happening is that if a source package is installed, pkgr builds the binary and caches that built binary. A subsequent call to pkgr install appears to simply install this binary. Is that what is actually happening?

copernican avatar Dec 04 '20 13:12 copernican