Propagating env vars

[andrewchambers]

I think we need a helper function for propagating env vars across packages. I can think of a few types of paths we might want to propagate:

• LD_RUN_PATH
• PATH
• PKG_CONFIG_PATH
• JANET_PATH

One way we can do this may be writing a file $PKG/env.jdn

The contents might look like this:

{
  "PATH" : "..."
  "PKG_CONFIG_PATH" : "..."
  "LD_RUN_PATH" : "..."
}

We can then write a helper functions like:

(load-pkg-env-vars [liba libb libc])

Which would load and merge all of the paths.

@sogaiu Do you think something like that would help your packaging of X11?

I am hesitant to make it mandatory for all packages, but maybe its a useful convention for some packages.

[andrewchambers]

It seems like a function like this could also do lots of the job of manually set PATH.

[sogaiu]

For PATH it seems like it could be convenient, though for some reason I'm a bit wary that undesirable things might "leak in". May be it's an unwarranted concern.

For xorg, if PKG_CONFIG_PATH could be propagated, perhaps that would mean that symlinks to .pc files wouldn't need to be created (as discussed in https://github.com/andrewchambers/hpkgs/issues/90)?

I wonder whether one would always want to load / merge all env vars that are in env.jdn -- this is just a hunch.

This sort of functionality seems powerful (and helpful) -- though it seems like it could lead to breakage / change so may be the motivation for build artifact testing of some sort would be increased?

[andrewchambers]

This is making me understand some more of what Nixos does. They have some env vars that should be propagated to runtime, some things that should be propagated to build time only.

I wonder whether one would always want to load / merge all env vars that are in env.jdn -- this is just a hunch.

Maybe not. The loading function could take options.

[sogaiu]

Ah, I think it may be starting to get through a bit more.

For st, I guess it might be possible to arrange for some kind of "merged" env vars in addition to PKG_CONFIG_PATH -- specifically, one for xorg headers and another for xorg libraries. If that worked, then may be transitive dependency references could be eliminated from st's .hpkg.

[sogaiu]

I looked at my draft of emacs.hpkg that uses xorg and I think I see a similar kind of pattern to the one(s) in st.

It also has things like:

    (os/setenv "CPPFLAGS"
               (string *default-cflags*
                       " "
                       (string/join
                        (map (fn [lib]
                               (string " -I"
                                       (lib :path) "/include"))
                             [fontconfig
                              freetype2
                              libxcb
                              libpng
                              libICE
                              libSM
                              libX11
                              libXau
                              libXaw
                              libXext
                              libXft
                              libXmu
                              libXpm
                              libXrender
                              libXt
                              ncurses
                              util-linux                              
                              xorgproto
                              zlib]))))

[sogaiu]

I think as an initial version, something like:

(get-vars libA ["var-name1" "var-name2" ... "var-nameN"])

that returns a struct with:

{"var-name1" value1
 "var-name2" value2
 ...
 "var-nameN" valueN}

where the values are retrieved from env.jdn as you suggested, would be quicker and simpler to implement and would give more control on the calling side.

I guess we'd want some kind of corresponding saver. May be something like:

(set-vars {"var-name1" value1
           "var-name2" value2
           ...
           "var-nameN" valueN})

Merging can be implemented in prelude or one's .hpkg and retrieving from multiple libraries can just be done iteratively.

As a further refinement, we could just have get-vars return a struct without allowing any selection by var-name via the API, so just:

(get-vars libA)

...as selection can also be done in an .hpkg after the whole struct has been retrieved.

We can experiment sooner and determine what we really need without having to keep changing hermes proper -- if it turns out we always (or almost always) need merging or some other features, we can consider putting that in hermes once we have a better idea.

If it turns out this is all we need, hermes stays simpler too.

[sogaiu]

I guess one location in the source that might be close to where some of this would go is: https://github.com/andrewchambers/hermes/blob/master/src/pkgstore.janet#L605_L614

Does that seem right?

[andrewchambers]

I was thinking this could just be part of hpkgs and doesn't necessarily need to be built in.

[sogaiu]

Are there not potential trust issues?

For example, suppose some source code puts in place a file named env.jdn as part of its typical make install?

[andrewchambers]

The act of running a projects makefile already seems like it has more trust than writing a vars.jdn or env.jdn file.

I'm not quite sure why a file like this would be more trusted than the files already installed via make install?

[sogaiu]

Perhaps I was unclear in my original description, if so, sorry for the confusion. What I am trying to say has to do with a packaging author's perspective and idea of trust of information for the purpose of writing a package's definition, not necessarily that of the end user.

My current impression is that if hermes provides the result of the call for fetching the information (which is in turn the result of hermes being the one to have done the storing in a manner like we have been discussing) a package author can trust that another package author knowingly requested that specific information be made available to other package authors, not the upstream source.

It's true that most package definitions have an invocation akin to make install that likely results in files and directories that the package author is unware of ending up in a build's artifact. But as a package author, I think that another package author is also not fully aware and thus I don't / wouldn't rely on the info in the same way I would for the info in say, .hpkgs.jdn. I know that hermes figured that stuff out and wrote it there -- if a package definition creates a file by that name, AFAICT it is overwritten by hermes.

Does this make any more sense? May be I am missing something...