nimble icon indicating copy to clipboard operation
nimble copied to clipboard

nimble.ini with only nimbleDir fails

Open c-blake opened this issue 6 years ago • 19 comments

Nothing works with a ~/.config/nimble.ini like:

nimbleDir = "/my/home/.config/nimble"

where /my/home should be the user's home directory. E.g.,

nimble install --verbose ANYTHING
    Reading config file at /my/home/.config/nimble/nimble.ini
     Error: Package list '' requires either url or path

Providing a PackageList section pointing to the official repository fixes things, but should not be necessary, IMO.

c-blake avatar May 11 '18 09:05 c-blake

This could not be reproduced with latest nimble.

genotrance avatar Jul 19 '20 23:07 genotrance

It is true that things do currently work better than when this issue was opened in that nimble.ini is not necessary for regular nimble functioning out of $HOME/.nimble.

However, without ~/.config/nimble/nimble.ini (but with an otherwise fully populated ~/.config/nimble) nimble still spams the top-level of my home dir with ~/.nimble and tries to operate out of there. So, it won't know about previously populated ~/.config/nimble/pkgs, for example, but yeah, it can at least install new things.

It seems to me that if nimble can detect ~/.config/nimble/nimble.ini and switch to the XDG friendly mode, then it should just switch to that mode if ~/.config/nimble/ exists at all. Then users can just opt-in to the XDG via mkdir ~/.config/nimble. This is how many other XDG-sensitive-but-non-defaulting applications work.

Instead nimble still requires a new file with a non-trivial to guess name (nimble.ini) with specific non-trivial to guess contents like nimbleDir = "MY-HOME/.config/nimble".

I can edit the title of the issue if you want, but there is still something "real" to address here.

c-blake avatar Jul 20 '20 08:07 c-blake

This can be done fairly easily. The only minor problem is that anyone who has run nimble test has created a nimble.ini in ~/.config/nimble as part of the testing. It doesn't contain nimbleDir = so doesn't affect users today but once this change goes in, suddenly these users will see their nimbleDir automatically move and the original ~/.nimble will be ignored.

@dom96 - open to your feedback on this one.

genotrance avatar Jul 20 '20 15:07 genotrance

It sounds like nimble test creating some shadow ~/.config/nimble/nimble.ini (in addition to presumably another functional one in ~/.nimble/nimble.ini?) is a bad situation. Why does it do that? Which nimble.ini even gets used when both exist? If not a question for the program, surely a question a user would ask.

c-blake avatar Jul 20 '20 16:07 c-blake

Nimble does not look for the ini file in ~/.nimble/nimble.ini, only in the XDG config directory.

genotrance avatar Jul 20 '20 17:07 genotrance

Ok. I could still see any user asking "which?", even if the program only looks for one, especially if the program is auto-creating (for pure ~/.nimble users) this weird .config/nimble/nimble.ini hierarchy. Or does it only create .config/nimble/nimble.ini if ~/.config already exists? Or .config/nimble? That all sounds like kind of an unnecessary tangle of questions and issues for both users and the program.

I don't even know what nimble.ini is for..besides my one use which is to make ~/.config/nimble work out. The truth is that I barely use nimble except to test that it works for my own packages once in a while.

Anyway, I stand by my recommendation that if ~/.config/nimble exists just set some global mode/path prefix to always use it..and now expand that recommendation to always honor that global when creating any new files. So, if nimble test needs to create any new files it should be under ~/.nimble or under ~/.config/nimble according to that global setting.

Another thing people do along these lines is to globally key off of an environment variable like $NIMBLE or $NIMBLEDIR.

Anyway, I'm just trying to be helpful. I don't know what kind of backward compatibility questions/issues intersect with all this, but having two home dir namespaces for one program seems pretty bad to me.

c-blake avatar Jul 20 '20 19:07 c-blake

I'm so confused.

The only minor problem is that anyone who has run nimble test has created a nimble.ini in ~/.config/nimble as part of the testing.

Are you saying that nimble creates a nimble.ini file?

dom96 avatar Jul 20 '20 19:07 dom96

That's what @genotrance said. As mentioned, I'm not sure under what precise set of conditions (EDIT: or why it creates it).

c-blake avatar Jul 20 '20 19:07 c-blake

Again, it is only created by nimble test and not deleted.

Nimble doesn't create it for regular users so this really only affects developers of nimble.

genotrance avatar Jul 20 '20 19:07 genotrance

As mentioned, I don't really use nimble, but I thought nimble test was something any package developer, not just a nimble dev, ran to invoke tests within their package.

c-blake avatar Jul 20 '20 19:07 c-blake

Ah, I see the confusion now - I meant nimble test of Nimble itself since its tester verifies that the nimble.ini functionality works correctly. If you run nimble test on your package, it doesn't do this.

Sorry for the confusion.

genotrance avatar Jul 20 '20 19:07 genotrance

Ah. That sounds much less bad. :-) If nimble devs just need to remember to clean up their own tests that isn't much of a reason to not do something, but @dom96 can/should weigh in.

Anyway, I already figured out how to ~/.config/nimble work. This is for other people who could maybe "just guess" if they mkdir .config/nimble then it might work. Other stuff is kinda search through GH issues/docs/dark arts/not so obvious/without precedent in other tools.

c-blake avatar Jul 20 '20 20:07 c-blake

Well, so Nim moved its cache from ~/.nimcache to ~/.cache/nim so it seems we should just move to ~/.config/nimble by default and stop using ~/.nimble altogether. Do packages qualify as configuration or is ~/.local/lib/nimble` more appropriate?

And what about Windows?

genotrance avatar Jul 20 '20 20:07 genotrance

We should definitely not move the default Nimble package install directory to ~/.config/nimble. What is the actual problem here that is being discussed? You can tell Nimble to install packages into wherever you want, there is no need to change the default install directory.

dom96 avatar Jul 20 '20 20:07 dom96

Python's pip uses ~/.local for packages, but I think that's actually very rare (I honestly don't know of any besides pip!), and because of that I kinda wish it just used ~/.config. I think it's kind of debatable, but there is a big specification (https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) for this stuff that some people get pretty passionate about with regards to NFS and symlinks and backup space and that sort of thing.

Anyway, I am fine with a simple run-time check for ~/.config/nimble and switching off of either that or an environment variable. Or even doing nothing. I mostly chimed in here when it was said the problem could not be reproduced and I thought I could maybe help less clueful people. :-)

c-blake avatar Jul 20 '20 21:07 c-blake

Python's pip uses ~/.local for packages, but I think that's actually very rare (I honestly don't know of any besides pip!)

Rust uses ~/.cargo and Haskell uses ~/.cabal. I bet there are many others that do the same.

and because of that I kinda wish it just used ~/.config.

You want your package to be installed into a directory specifically designated for holding configuration files? Really?

dom96 avatar Jul 20 '20 21:07 dom96

I literally only have one application (ssh) that uses a raw $HOME entry and they are adamant and will never change. The rest is under ~/.config or ~/.cache and, for good or ill, those are the two parts of the XDG spec that practically anyone has paid attention to. So, at least for me, ~/.local is only for Python, if I used pip which I do not. Zathura the pdf reader for example keeps its "input-history" (your searches like .lesshst) in its config dir. And the list would go on and on and on were I to construct it. If I had more than one reason for ~/.local then sure I'd prefer it, but if it's only one reason the directory is doing no separation work. So, it may as well be ~/.nimble if you put it there.

But really I am just answering your question since you seemed incredulous. XDG covers all of that (and really so much more that like I said it gets ignored) with environment variables that tell applications where to put all the various directories. So, if you actually follow that spec, I should be able to merge .local/nimble into my .config/nimble just with an evar. (EDIT: In fact, I could probably make pip not create ~/.local with evars and I probably would if I actually used it for more than test cases here & there.)

c-blake avatar Jul 20 '20 21:07 c-blake

Linking to the spec: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

Looks like the packages really belong in $XDG_CACHE_HOME which would be ~/.cache/nimble. Ini file is searched in ~/.config/nimble already, would need to update that code to check $XDG_CONFIG_HOME env var as well to be compliant with the spec.

From XDG point of view, putting packages user specific non-essential data files like packages in config seems incorrect.

genotrance avatar Jul 22 '20 01:07 genotrance

It probably comes down to interpretation of "essential". I think one valid set up for $XDG_CACHE_HOME is a symlink it into a RAM filesystem that is lost on any reboot..So, hard-wiring a RAM-only caching. Such volatile, non-NFS/etc. shared storage may make sense for nimble's packages_*.json, but a reboot wiping out bin/ and pkgs/ is probably undesirable.

The XDG spec is incomplete anyway, as regards this kind of $HOME-sharing unfriendly package manager operating out of $HOME business nimble is in. As far as I know XDG does not address incompatible binary files at all..i.e. CPU specific but non-volatile essential files. So, suppose you had -march=native flags in your nim.cfg|config.nims to get say AVX2 or AVX512 instructions. Then if you built a binary on one computer having those instructions, the created ~/.config/nimble/bin/foo would not be sharable with some older CPU..So, you would want that part of $HOME not-shared and that is more like ~/.cache as you mention, only maybe non-volatile/non-backed up/expensive to recover as I mentioned in the first paragraph.

If pkgs/ were also in ~/.cache, and is the only record of "what's installed" then it might be impossible to recover. pkgs/ seems in nimble to be sort of both user configuration and user data, but I'm not sure what @genotrance meant by simply "packages".

Anyway, it's a complex morass of questions. Just doing $XDG_CONFIG_HOME/nimble for everything seems reasonable to me. You just have one top level root ~/.nimble or ~/.config/nimble. This is what about 90% of other unix apps seem to do (and probably 99% of unix apps that also allow just $HOME/.appname). The XDG spec is one of those "only partly obeyed" things. I just thought there should be an easier way to activate that mode than having to guess both nimble.ini, and its contents.

c-blake avatar Jul 22 '20 10:07 c-blake