Use a global package cache

[Pauan]

I'm not sure whether this belongs in this repository or purescript/purescript

Right now, psc-package downloads packages and places them into the .psc-package folder:

.psc-package
└─┬ psc-0.10.2
  └─┬ prelude
    └─┬ v2.1.0
      └── ...

Instead, I propose for it to install packages into a single cache folder, and then create a symlink in .psc-package:

.psc-package
└── psc-0.10.2 -> $XDG_CACHE_HOME/psc-package/package-sets/psc-0.10.2

$XDG_CACHE_HOME
└─┬ psc-package
  ├─┬ packages
  │ └─┬ prelude
  │   └─┬ v2.1.0
  │     └── ...
  └─┬ package-sets
    └─┬ psc-0.10.2
      └── prelude -> $XDG_CACHE_HOME/psc-package/packages/prelude/v2.1.0

Notice that there are actually two symlinks:

• .psc-package/psc-0.10.2 symlinks to $XDG_CACHE_HOME/psc-package/package-sets/psc-0.10.2

  The reason is: if multiple PureScript programs use the same package set, it only needs to be downloaded once.

• $XDG_CACHE_HOME/psc-package/package-sets/psc-0.10.2/prelude symlinks to $XDG_CACHE_HOME/psc-package/packages/prelude/v2.1.0

  The reason is: if the same package+version is included in multiple package sets, it only needs to be downloaded once.

This has a few benefits:

• It saves a non-trivial amount of internet bandwidth.

• It saves time, since the files do not need to be downloaded/unzipped/untarred/copied repeatedly.

• It saves disk space, since only a single copy of a package or package set needs to exist (per version).

• It avoids unnecessary file writing, which is important on SSDs.

P.S. It's probably a good idea to make everything inside of $XDG_CACHE_HOME/psc-package read-only. If it is read-write, and if somebody modifies their local .psc-package folder, then it will affect all PureScript programs on their computer (because of the symlinks). This is very unexpected behavior.

P.P.S. This proposal uses the $XDG_CACHE_HOME env var, which defaults to $HOME/.cache/. That is the correct location to use on *nix. Windows should probably use %LOCALAPPDATA% and Mac should probably use ~/Library/Caches/

You might find this useful, though it does not seem to support Mac.

[paf31]

Yes, we've talked about this before, thanks for writing it up here.

I like the symlink idea too, although it might be tricky on Windows?

[garyb]

Windows has symlinks analogs, so I don't see why it wouldn't work. bower link still uses them, for example.

[paf31]

Ah ok, thanks. I haven't used Windows for a while now, so I wasn't sure 😄

[Pauan]

@paf31 @garyb Apparently symlinks in Windows have various issues:

http://security.stackexchange.com/questions/10194/why-do-you-have-to-be-an-admin-to-create-a-symlink-in-windows

http://superuser.com/questions/10727/why-cant-normal-users-on-windows-create-symbolic-links

http://blog.rlucas.net/rants/dont-bother-with-symlinks-in-windows-7/#comment-735

However, it may be possible to use NTFS junction points instead, which seems to have a similar purpose, but without the same problems:

https://devtidbits.com/2009/09/07/windows-file-junctions-symbolic-links-and-hard-links/

Unfortunately I don't have much experience with Windows, so I can't help much with that.

[garyb]

Yeah, junction points is what I was talking about :)

[hdgarrood]

Possibly worth considering simultaneously: https://github.com/purescript/purescript/issues/2477.

[paf31]

Might be worth noting that users can actually set this up manually right now, by creating the symlink themselves, which makes me think that it doesn't need to exist as the default functionality in psc-package. It might be good to add a flag to customize the location of the .psc-package directory though.

Maybe this should move to psc-package.