rattler-build icon indicating copy to clipboard operation
rattler-build copied to clipboard

feat: delay jinja evaluation for script

Open wolfv opened this issue 1 year ago • 4 comments

This delays the Jinja evaluation for the script contents. That means that we can also inject a lot of variables into the script by using jinja.

Such as ${{ PREFIX }} or ${{ PYTHON }}.

For example the following two examples work on Windows and on Unix:

build:
  script: cargo install --root ${{ PREFIX }} .
  # or
  script: ${{ PYTHON }} -m pip install .

These variables are also available as environment variables. However, since we are using cmd.exe and bash on the different operating systems, the syntax to expand environment variables is not the same (%PREFIX% vs $PREFIX).

Alternatives we considered (or are considering):

  • add a jinja function env("PREFIX") that expands to either %PREFIX% or $PREFIX depending on execution context. Might not work well with different interpreters.
  • automatically convert $PREFIX to %PREFIX% as a string replacement on cmd.exe

TODO

  • [x] missing values from context
  • [ ] Adjust the recipe rendering CEP because now we need the variables from the Context later as well

wolfv avatar May 31 '24 13:05 wolfv

Something that we might want to think about is that this does not solve the case for e.g. CMake or Meson that both have ${MESON_ARGS} or ${CMAKE_ARGS} defined in their activation scripts that make them work quite OK cross-platform.

wolfv avatar Jun 02 '24 15:06 wolfv

We could still add the env function, e.g.:

script:
  - cmake ${{ env("CMAKE_ARGS") }} .
  - ninja install

wolfv avatar Jun 02 '24 15:06 wolfv

Or we change the way we do activation completely (to work like pixi) and add everything to the Jinja context :)

wolfv avatar Jun 06 '24 09:06 wolfv

Or we change the way we do activation completely (to work like pixi) and add everything to the Jinja context :)

To me, from a user perspective, this makes the most sense. Adding all the environment variables after activation to the env jinja object that is.

It might also be slightly confusing to users which variables are available when but at least we provide a consistent way to access environment variables.

baszalmstra avatar Jul 18 '24 12:07 baszalmstra

Would this have an expanded path hard-coded when the recipe is rendered? That would make rebuilding a conda package like with reproducible-builds impossible.

isuruf avatar Sep 20 '24 16:09 isuruf

Actually, it's a requirement for the reproducible builds to build the packages in the exact same prefix regardless, so that's not really an issue. A lot of packages ship files with the prefix in some text files (such as .pc files etc.) and e.g. Debian also builds packages in the same folder always afaik. So I hope that being stable against changes in the filesystem location is not a requirement for reproducible builds.

wolfv avatar Sep 20 '24 17:09 wolfv