cabal
cabal copied to clipboard
haskell
adopt XDG basedir spec
(Imported from Trac #688, reported by guest on 2010-05-12)
I want to be able to tell cabal to read the configuration from a different place than $HOME/.cabal/config.
The ultimate goal is to get rid of the non-standard .cabal directory altogether, so far careful --package-db/--prefix etc. juggling served me well; config is the last remaining file there.
> cabal --version cabal-install version 0.8.2 using version 1.8.0.2 of the Cabal library
(Imported comment by @dcoutts on 2010-05-12)
I'm not quite sure why you say it is non-standard. It's the standard unix place to put per-user application files. Where would you consider to be a standard location?
You can use:
cabal --config-file=... [command] [flags]So it should work to use a shell alias, something like
alias cabal="cabal --config-file=..."So really the bug here is not that it's hard coded (since it's not) but that these global flags are not easily discoverable. We don't list them all in the global --help so as not to clutter things, but then we need a way to let people know that there are extra global options and provide a way to get the help for them via some extended --help thing.
So let us know about:
- what you think the standard location should be
- if that shell alias (or whatever equivalent you choose to use) works ok
(Imported comment by guest on 2010-05-13)
Sorry for neglecting to answer right away, I forgot about this bug for a few days because I didn't bother to jump through the hoops to be able to receive change notification.
If by standard you mean de facto just everyone's been doing it like that, then I agree. And since indeed everyone's been doing that, I end up with hundreds of dotdirs in my home directory. This sucks because their content is essentially unstructured. The problem is that I want to treat files differently according to their purpose; and there are not just application files. There are also configuration files, they are put under version control. There are cache files, they are excluded from backup and indexing.
You know that a software package in Linux is not just dumped into a single directory somewhere, but the files are sorted into directories which each have a certain meaning. FHS governs the semantics for the system layout, FDO's XDG base directory specification governs this for a user's directory. These are real standards, and they enable all sorts of interoperability that would not be possible with unstructured directory contents.
Whether you want to adopt the spec in order to sort files into their appropriate XDG location by default is up to you. Thanks to your hint about --config-file I was able to move away the last file and delete .cabal for good, so certainly a compliant layout is already possible (through careful configuration, as I said before) if a user wants it.
(Imported comment by @dcoutts on 2010-05-17)
I thought you might say the XDG spec. Yes I realise it's a "real" standard as opposed to a de facto standard. I must admit I'm slightly sceptical about the attempt to redefine where apps keep their per-user info. So far, very few apps seem to follow this new spec (mainly a few GNOME desktop bits). I will be surprised if they ever manage to get more than the two main desktops to move their dot files.
That said, if the world is going to move that direction then we don't want to get in the way. We should make it easier for users or distros to change where the per-user files go by default.
Thanks for confirming it is at least possible (if not convenient).
(Imported comment by guest on 2010-05-17)
we should open another ticket on providing extended help for the global flags
This is now tracked in #690.
(Imported comment by @dcoutts on 2010-05-17)
Hmm, I notice the XDG base directory specification does not say anything about program executable files or library files, though the default naming scheme sort-of suggests ~/.local/bin and ~/.local/lib.
(Imported comment by Syzygies on 2010-05-17)
I found this ticket while researching precedents for a related question.
For various reasons I like to build multiple versions of GHC side by side: Migration between major releases, testing candidate releases. For torture-testing new hardware, building many copies of GHC at once is better than e.g. the usual prime testing. I once literally caused a server power supply to smoke this way; my friend's hardware vendor was in over their head. They took my GHC test in-house to minimize delivery cycles.
A related piece of de facto convention comes up here: It is crazy to sprinkle pieces of a system like GHC into separate global locations like /usr/local/bin, /usr/local/share, unless one will only ever use one version, with no desire to look under the hood or move the installation. In other words, I have no issue with Linux or OS X shipping this way, but major additions should be localized by default. Some are, e.g. TeX. GHC should.
Fortunately, to avoid Siamese twin installations with accidentally shared parts, one can use the ./configure --prefix option while building GHC. Then, it becomes apparent that "cabal install" is the remaining holdout to ideal version localization. There is a perfect storm of tiny factors keeping Cabal from gracefully localizing. I wanted to elicit advice before contributing a fix; this ticket seems to have touched on a key issue.
I did not know about the "--config-file=..." command-line option; I will start using it in scripts. However, we need a better default for global installs.
In /usr/local I now use a symbolic link from ghc to one of the directories ghc-6.10.4, ghc-6.12.1, ghc-6.12.2, ghc-6.12.3-rc1. It then appears that I built GHC using "--prefix=/usr/local/ghc". My ~/.cabal/config file includes lines like "remote-repo-cache: /usr/local/ghc/packages". This works, but I've had amusing debates with unix experts on why my symbolic link is a terrible solution. Basically, when I change this link I pull the rug out from under any ongoing process that wants to see a consistent environment.
Environment variables are one "right" way to handle this. In short, a symbolic link is like changing state, while environment variables are like functional programming.
However, not only does .cabal/config fail to allow external environment variables, it doesn't allow forms like one sees in its own comments. (Who reads documentation? We all read sample code.) For example, even though I see the comment "-- bindir: $prefix/bin" I cannot write "remote-repo-cache: $prefix/packages". I would prefer to be able to write anything that would evaluate correctly in a shell to an absolute path, e.g. which ghc or perl -pe ... and so forth.
Ideally, a global install would by default use a global location for the Cabal config file, but we slam back into the original problem: By default the global install of GHC did not use the --prefix option, so there is no well-defined location parametrized by the version of GHC. On the other hand, this isn't so bad, e.g. put everything in ../share/hackage.haskell.org relative to the currently visible ghc. Saves bandwidth caching packages; if someone hasn't bothered to build ghc using --prefix, they won't mind the Siamese twin effects with Cabal either.
The simplest change that would help my issue would be to allow both absolute and relative paths in .cabal/install, with a relative path relative to the location of which ghc. This would "just work" in simple cases, and allow competing processes to coexist, that chose different versions of ghc via a custom $PATH alone, with no other flags needed. If this change was accepted, I wouldn't mind leaving my config file in ~/.cabal. That location is only mildly annoying; as has been noted, we've already lost the war on keeping that area uncluttered.
However, it just shouldn't matter which authorized user works on a global install. Relying on ~/.cabal/config for a global install should be viewed as a bug, as it prevents several admins from cooperating in consistently maintaining GHC on a shared system.
(Imported comment by @dcoutts on 2010-05-28)
Replying to Syzygies:
A related piece of de facto convention comes up here: It is crazy to sprinkle pieces of a system like GHC into separate global locations like /usr/local/bin, /usr/local/share, unless one will only ever use one version, with no desire to look under the hood or move the installation. In other words, I have no issue with Linux or OS X shipping this way, but major additions should be localized by default. Some are, e.g. TeX. GHC should.
You can already do this if you want to. You just set the --prefix to be what you want, e.g. instead of using /usr/local as the prefix, use /opt/ghc/6.12.1 or whatever, then you'll get paths like /opt/ghc/6.12.1/bin, /opt/ghc/6.12.1/lib, /opt/ghc/6.12.1/share etc etc.
Fortunately, to avoid Siamese twin installations with accidentally shared parts, one can use the ./configure --prefix option while building GHC. Then, it becomes apparent that "cabal install" is the remaining holdout to ideal version localization. There is a perfect storm of tiny factors keeping Cabal from gracefully localizing. I wanted to elicit advice before contributing a fix; this ticket seems to have touched on a key issue.
I did not know about the "--config-file=..." command-line option; I will start using it in scripts. However, we need a better default for global installs.
The "default" default, should follow global conventions, meaning using /usr/local. Again you can override the prefix and all other install directories.
It's true that there is not currently a sensible way to have a global cabal config file, in the style of /etc/cabal or something like that. Following the XDG spec in that regard might make some sense.
Environment variables are one "right" way to handle this. In short, a symbolic link is like changing state, while environment variables are like functional programming.
You mean you would like an environment variable to set the location of the Cabal config file? You can set the $CABAL_CONFIG, it acts like setting the "--config-file=..." option.
However, not only does .cabal/config fail to allow external environment variables, it doesn't allow forms like one sees in its own comments. (Who reads documentation? We all read sample code.) For example, even though I see the comment "-- bindir: $prefix/bin" I cannot write "remote-repo-cache: $prefix/packages". I would prefer to be able to write anything that would evaluate correctly in a shell to an absolute path, e.g. which ghc or perl -pe ... and so forth.
The prefix is something that makes sense in the context of an installation. It is not clear that basing the cache location on that makes sense. It means for example that it will not find the default cache if you override the --prefix on the command line. Perhaps what you really want to do is to specify the default prefix and cache dirs in the config file based on some other variable, rather than having the cache dir follow the prefix.
Ideally, a global install would by default use a global location for the Cabal config file,
Do you mean an install run as root, or do you mean an install run as a user? Or do you mean that you'd like to be able to have a shared/global set of connfig defaults that can be overridden in per-user config files?
As a user when you run cabal install --global --su-cmd=sudo you're using your own config, but installing to a global location (the exact prefix etc is set in the config file). It does the build as user and installs as root.
but we slam back into the original problem: By default the global install of GHC did not use the --prefix option, so there is no well-defined location parametrized by the version of GHC.
You can either use versioned ghc binaries, (called ghc-6.10.4, ghc-6.12.1 etc) in a shared prefix like /usr/local or as I described above you can use separate prefixes for different ghc installations. Note that by default, cabal installs libs in dirs including the ghc version, so it all works. As mentioned before you have complete control over the prefix (and other detailed layout of bindir etc) so you can put files where you like, according to some ghc version scheme. You can use the compiler id in the cabal prefix for example.
The simplest change that would help my issue would be to allow both absolute and relative paths in .cabal/install, with a relative path relative to the location of which ghc.
Honestly I don't think that is sensible. Allowing people to use more variables so that you can specify a prefix that happens to correspond to your ghc installation is one thing, making that the meaning for relative paths is rather another. Usually you do not want to install things into the ghc tree, so using relative paths would encourage that dangerous behaviour.
However, it just shouldn't matter which authorized user works on a global install. Relying on ~/.cabal/config for a global install should be viewed as a bug, as it prevents several admins from cooperating in consistently maintaining GHC on a shared system.
I don't understand this bit. Perhaps you can explain what config you would like to use for global installs. Where would it live? Do you assume global installs are only performed by root?
I think it would help if you could explain what you're actually trying to achieve. The mailing list might be the better place for that discussion.
(Imported comment by @kosmikus on 2010-05-28)
Moving to 0.16 milestone so that we look at it again sometime soon, but I think this is a candidate for | (meaning that without external concrete patches or suggestions, we probably won't do it).
What is planned now about FreeDesktop XDG basedir specification for cabal?
http://ploum.net/post/207-modify-your-application-to-use-xdg-folders https://live.gnome.org/GnomeGoals/XDGConfigFolders http://standards.freedesktop.org/basedir-spec/latest/
According to XDG Base directory specification, cabal should not have its own folder (.cabal) anymore User data should go into $XDG_DATA_HOME (which default to ~/.local/share), user preferences should go into $XDG_CONFIG_HOME (which default to ~/.config) and cached data should go to $XDG_CACHE_HOME (which default to ~/.cache). More details at : http://ploum.net/post/207-modify-your-application-to-use-xdg-folders https://live.gnome.org/GnomeGoals/XDGConfigFolders
Full specification can be found at: http://standards.freedesktop.org/basedir-spec/latest/
The Freedesktop.org XDG base directory specification have good de facto adoption. It has been adopted by:
- GNOME ( https://live.gnome.org/GnomeGoals/XDGConfigFolders )
- GTK+ ( https://bugzilla.gnome.org/show_bug.cgi?id=646631 )
- KDE ( http://techbase.kde.org/KDE_System_Administration/XDG_Filesystem_Hierarchy#Freedesktop.org_and_Standard_Directories )
- QT ( http://harmattan-dev.nokia.com/docs/library/html/qt4/qsettings.html#setPath )
- XFCE ( http://docs.xfce.org/xfce/xfce4-session/advanced in Files and Environment Variables )
- LXDE
- Razor-qt
- VLC ( https://trac.videolan.org/vlc/ticket/1267 )
- GStreamer ( https://bugzilla.gnome.org/show_bug.cgi?id=518597 )
- Chrome ( http://code.google.com/p/chromium/issues/detail?id=16976 )
- many more upstream applications
- Ubuntu ( http://brainstorm.ubuntu.com/idea/6557/ & http://packages.ubuntu.com/fr/source/precise/libxdg-basedir )
- Debian ( http://packages.debian.org/squeeze/libxdg-basedir1 )
- Red Hat
- Fedora
- Suse
- many more distributions
I think that cabal should use same locations than the vast majority of Desktop environment and applications.
There are real advantages of following this specification :
- a lot less cluttered $HOME
- Make backups a lot more safer and easier. Backuping your $XDG_DATA_HOME along with your files is enough (or just excluding $XDG_CACHE_HOME)
- A lot easier to reset a default configuration if you want/need it (and without any risk to loose informations). Even for the software itself could choose to reset $XDG_CONFIG_HOME if needed.
- Avoid some strange bugs that happens because you had a old version of some configuration file
- A lot more of flexibility and portability because no path are hardcoded.
In general, I tend to agree with these advantages.
However, the XDG spec seems to be designed with typical desktop application in mind, not compilers or package managers. As @dcoutts said above, it doesn't specify where the binaries, libraries, documentation should go. Are there any examples of programming language implementations that use this scheme?
Also, should we try to follow the same spec on Windows as well? (I guess there are no technical reasons not to do so.)
Users binaries, documentation and libraries are users data and thus should go to $XDG_DATA_HOME (which default to ~/.local/share),
"Windows have the Local/Roaming split which could be roughly equated to putting everything in either $XDG_DATA_HOME or $XDG_CACHE_HOME like most Humble Bundle Linux game ports do."
Users binaries, documentation and libraries are users data and thus should go to $XDG_DATA_HOME (which default to ~/.local/share),
Yes, that's exactly what I meant by the desktop application oriented mindset.
Putting binaries, libraries and docs in one directory is a downgrade from the scheme that we have now.
Or would it be $XDG_DATA_HOME/bin etc.? Then the full path is ~/.local/share/bin, which looks weird and complicated.
I think it is better to avoid users specific binaries, libraries and doc and share these files between all users.
Whether it is better or not is out of the scope of this issue, I believe.
There is no $XDG_DATA_HOME/bin etc but you are free to use $XDG_DATA_HOME/cabal/bin etc
So that would be ~/.local/share/cabal/bin, which looks rather disappointing to anyone who ever cares about what these paths are.
I think at this point it would be more realistic for someone who cares to create a simple cabal-install wrapper that would conform to XDG. Then release it and see if it gains popularity. If it does, it would be a convincing argument to switch to that scheme by default.
Instead of conforming to XDG by default I'd be happy with cabal-install reading an environment variable telling cabal where its config is. Environment variables are much more persistent than some kind of flag+alias for that. Then you can configure all other directories through config anyway.
EDIT Ignore me, reading an issue I found $CABAL_CONFIG. I'd be great if it was documented though.
EDIT2 It'd be great if cabal config evaluated external environment variables as well.
Personally I think cabal-install should follow XDG here which implies ${XDG_CONFIG_HOME:-$HOME/.config}/cabal for {config,world} and ${XDG_DATA_HOME:-$HOME/.local/share}/cabal for {bin,lib,logs,share} and ${XDG_CACHE_HOME:-$HOME/.cache}/cabal for {packages,setup-exe-cache}. It can then, separately, symlink binaries to a configurable location defaulting to $HOME/.local/bin (no XDG), which isn't in any official standard I'm aware of but is used by Python (like PREFIX=$HOME/.local) and is on the user $PATH by default in at least Fedora.
I have no idea what those paths should map to on Windows systems but presumably that's a solved problem.
You realise that you guys can actually change the XDG_* environment variables? XDG_DATA_HOME doesn't have to be HOME/.local/share. We'll get there eventually, compilers are one of the few remaining bastions supporting the UNIX v2 bug that is "dotfiles"
@Earnestly Exactly. That is the default but can be changed as you like. So, @nagisa, $CABAL_CONFIGwould be the exact same thing as the XDG variables.
Another problem with using a unique environment such as CABAL_CONFIG is we
add to our already bloated environs. For reference here are a few:
export SLRNHOME=$XDG_CONFIG_HOME/slrn
export _RR_TRACE_DIR=$XDG_DATA_HOME/rr
export CARGO_HOME=$XDG_DATA_HOME/cargo
export GNUPGHOME=$XDG_CONFIG_HOME/gnupg
export INPUTRC=$XDG_CONFIG_HOME/inputrc
export RLWRAP_HOME=$XDG_DATA_HOME/rlwrap
export LYNX_CFG_PATH=$XDG_CONFIG_HOME/lynx
export DVDCSS_CACHE =$XDG_CACHE_HOME/dvdcss
export TIGRC_USER=$XDG_CONFIG_HOME/tig/tigrc
export WEECHAT_HOME=$XDG_CONFIG_HOME/weechat
export PERL_CPANM_HOME=$XDG_CACHE_HOME/cpanm
export GIMP2_DIRECTORY=$XDG_CONFIG_HOME/gimp
export ELINKS_CONFDIR=$XDG_CONFIG_HOME/elinks
export CABAL_CONFIG=$XDG_DATA_HOME/cabal/config
export HTTPIE_CONFIG_DIR=$XDG_CONFIG_HOME/httpie
export LESSHISTFILE=$XDG_CACHE_HOME/less/history
export XAUTHORITY=$XDG_RUNTIME_DIR/x11/xauthority
export XCOMPOSEFILE=$XDG_CONFIG_HOME/x11/xcompose
export GUILE_HISTORY=$XDG_CONFIG_HOME/guile/history
export VIMPERATOR_RUNTIME=$XDG_CONFIG_HOME/vimperator
export PENTADACTYL_RUNTIME=$XDG_CONFIG_HOME/pentadactyl
export NOTMUCH_CONFIG=$XDG_CONFIG_HOME/notmuch/notmuchrc
export GTK2_RC_FILES=$XDG_CONFIG_HOME/gtk-2.0/settings.ini
export TERMINFO_DIRS=$XDG_DATA_HOME/terminfo:/usr/share/terminfo
export UNCRUSTIFY_CONFIG=$XDG_CONFIG_HOME/uncrustify/uncrustify.cfg
These are just the ones I use, there are much more. Clearly this is not sustainable, each export adds to the total memory consumption of -every- child process on the machine.
Here are some more https://wiki.archlinux.org/index.php/XDG_Base_Directory_support#Partial
So no improvements here? Another reason for adopting the XDG scheme is that (some) backup tools know about these and already exclude e.g. .cache from backups. I'm now hunting around to see which of its 3GiB cabal actually needs :-(
@rkraneis I think this mostly needs somebody to take the lead: design a reasonable migration/fallback scheme, and implement it.
I was browsing this topic researching what the XDG file location concept says about binaries - not to much as it seems - and I found that @Earnestly wrote a few years back:
... We'll get there eventually, compilers are one of the few remaining bastions supporting the UNIX v2 bug that is "dotfiles"
however the original link was a Google+ one that has now been expunged from the Interweb thingummyjig - but I tracked down a wayback machine archive and fitted it into this post. HTH a tiny bit...
Until PRs are done, I updated the entry in the Arch wiki with easy environment variables ;) https://wiki.archlinux.org/index.php/XDG_Base_Directory#Partial
I'm idly poking at this now. Did someone ever draft a design or migration path? Let me suggest the following:
- The Cabal configuration file goes in
$XDG_CONFIG_HOME/cabal/config. cabal installbinaries go in$HOME/.local/bin(from the systemd file-hierarchy spec).- Everything else goes in
$XDG_CACHE_HOME/cabal).
This might break people, of course. So if CABAL_DIR exists, then the current behaviour will be maintained (everything in CABAL_DIR). Further, as a (hopefully temporary) migration path, if $HOME/.cabal exists, then that will interpreted as CABAL_DIR being set to $HOME/.cabal.
I have no idea what the situation is on Windows. Maybe current behaviour should just be preserved there.
Something not mentioned previously. Users are able to configure using CABAL_CONFIG and having user specific paths in their cabal.config. E.g. mine looks like:
repository hackage.haskell.org
url: http://hackage.haskell.org/
remote-build-reporting: anonymous
remote-repo-cache: /cabal/packages
write-ghc-environment-files: always
build-summary: /cabal/logs/build.log
extra-prog-path: /cabal/bin
installdir: /cabal/bin
logs-dir: /cabal/logs
store-dir: /cabal/store
symlink-bindir: /cabal/bin
world-file: /cabal/world
install-dirs user
prefix: /cabal
Changing defaults is more or less a communication issue (EDIT: i.e. not a technical problem)
I don't think the code for that will need to be modified. XDG migration is only about the defaults, anyway. From hacking a bit on the code, I think the largest unknown challenge is that there seems to be some parts of the code that assume a single "root" cabal directory from which other paths can be computed. I'm not yet sure how deep that goes.