haskell
`cabal v2-install`ed executables should be resilient against (partial) deletion of `.cabal/store`
According to #6506, the default --install-method on Windows is copy instead of symlink.
I think this is the more sensible default in general.
From v1-install I am used to get binaries in .cabal/bin that persist even if I clean up in .cabal the artifacts of old GHC versions. With the defaults of v2-install, cleaning up old GHCs breaks my binaries, as they are only symlinks.
E.g., I have in .cabal/bin
hTags -> ../store/ghc-8.8.3/hTgs-0.1.4-060a447d/bin/hTags
which is dead since I purged the ghc-8.8.3 directory in store.
The new behavior is surprising to me and can be very annoying (e.g. in scenarios where I have an executable build by a old ghc version which I removed, but the compilation fails on the later ghc versions---it can take hours to recover a lost executable).
Proposal: Make --install-method=copy the default (instead of =symlink).
P.S.: Found on https://stackoverflow.com/questions/7167424/replace-all-symlinks-with-original a hack to replace symlinks with their target by abuse of sed:
~/.cabal/bin$ find . -type l -exec sed -i '' {} \;
(Does not preserve time stamps, though.)
Even with --install-method=copy executables aren't guaranteed to work when you delete .cabal/store because they may depend on data files or on other libraries if dynamic linking is enabled.
You may want #3333 instead Edit: or relocatable packages
Ok, but if I keep the while (lightweight) .cabal/share folder and use static linking (which is the default), then with default install-method: copy I should be able to keep working binaries.
It is the most sensible default I am after, so that things work out-of-the-box in standard cases for users like me that do not want to configure if they are not forced to.
.cabal/share
v2- doesn't use that directory anymore. v1- used a FHS-like hierarchy for ~/.cabal, but v2- switched to a nix-like hierarchy, where the full contents of a package (data files included) are in an isolated directory under ~/.cabal/store
Well, something is odd then in v2-cabal.
Once I have created a binary, it does not matter any more which ghc version I used to build it, so there shouldn't be any dependency on the ghc version any more. However, if some of its components are placed in a directory named after this ghc version, there is a dependency left.
So, with the current v2-cabal, what is the best practice to manage the store, to prevent it from eating up my hard drive? (I mean, thanks for pointing to #3333, but this isn't implemented yet. At the same time, v2-cabal is rolling out to replace v1-cabal, so it should have a plan how we can work with it sustainably.)
it does not matter any more which ghc version I used to build it, so there shouldn't be any dependency on the ghc version any more. However, if some of its components are placed in a directory named after this ghc version, there is a dependency left
There is:
> ldd .cabal/store/ghc-8.10.3/random-1.2.0-8751b6ab3bf8a29738d8affd64a32c530278d4d97e2b8b78157213eb4fd0fad3/lib/libHSrandom-1.2.0-8751b6ab3bf8a29738d8affd64a32c530278d4d97e2b8b78157213eb4fd0fad3-ghc8.10.3.so
linux-vdso.so.1 (0x00007fff7037f000)
libHSsplitmix-0.1.0.3-5263e456e3deb12ee532288e6660fa3141fb27d86848a24ef3f795b36bf28604-ghc8.10.3.so => /home/fgaz/.cabal/store/ghc-8.10.3/splitmix-0.1.0.3-5263e456e3deb12ee532288e6660fa3141fb27d86848a24ef3f795b36bf28604/lib/libHSsplitmix-0.1.0.3-5263e456e3deb12ee532288e6660fa3141fb27d86848a24ef3f795b36bf28604-ghc8.10.3.so (0x00007faa61845000)
libHSmtl-2.2.2-ghc8.10.3.so => /nix/store/2lkbmrf9iw3v4dnf0yixhldvff5397xl-ghc-8.10.3/lib/ghc-8.10.3/mtl-2.2.2/libHSmtl-2.2.2-ghc8.10.3.so (0x00007faa6180c000)
[...]
what is the best practice to manage the store, to prevent it from eating up my hard drive
Can't speak for everyone, but I just nuke a part of it every so often (and for me that often happens to match up with ghc releases and when I drop old ones). With a biggish drive it isn't much of an issue. I can see how it could become one when new-installing different versions of Agda (which is heavyweight) at a fast rate (one more reason to use cabal list-bin + ln -s in that case).
Really, I think the only safe way to free space without problematic edge cases here is #3333 (or maybe some way to safely relocate a package with all its runtime dependencies).
Back to the issue in the title, given the consequences of deleting the store, a symlink seems to me strictly better than a copy, would you agree?
I changed the title to reflect better the core problem of cabal v2-install (e.g. over cabal v1-install): It puts stuff (e.g. data files) into a GHC-version dependent location, and if you clean up that GHC version, the binary malfunctions.
An installed executable is independent of the compiler it was created with. The compilation infrastructure is needed to create the binary, not to run it.
Reproducer (full code see attached zip file):
import Paths_has_data_files (getDataDir)
main :: IO ()
main = do
dataDir <- getDataDir
putStrLn $ "Data directory: " ++ dataDir
contents <- readFile (dataDir ++ "/data-file.txt")
putStrLn contents
Demo:
$ cabal v2-install 1 х │ 10:01:25
...
$ has-data-files ✔ │ 10s │ 10:05:14
Data directory: /Users/abela/.cabal/store/ghc-9.12.2-ea3d/hs-dt-fls-0.0.0-fe9c2d75/share
Some data that comes with this application.
$ cabal v1-install
...
$ has-data-files ✔ │ 4s │ 10:05:53
Data directory: /Users/abela/.cabal/share/aarch64-osx-ghc-9.12.2-ea3d/has-data-files-0.0.0
Some data that comes with this application.
So the difference is that v2-install puts stuff in the fat store directory whereas v1-install puts it into the slim share directory.
In absence of a garbage collector for store (#3333) the only way to clean out old GHC version is to nuke (parts of the) store, destroying the applications I built with it.
In contrast, v1-install puts shared files into .cabal/share which is so small I never need to clean it up.
~/.cabal $ du -hd1
4.1G ./bin
...
15G ./store
13M ./share