grayskull icon indicating copy to clipboard operation
grayskull copied to clipboard

Question: why is --no-deps --no-build-isolation set?

Open henryiii opened this issue 1 year ago • 13 comments

Conda-build sets PIP_NO_BUILD_ISOLATION and PIP_NO_DEPENDENCIES (See https://github.com/conda/conda-build/blob/0ad4bb6829b440caae9ae8a4db14ab1ff3788ab7/conda_build/build.py#L3020). Why is it also being set in every recipe in https://github.com/conda/grayskull/blob/5b293078ea4fb17572dfa63ee1dde8612d884f00/grayskull/strategy/pypi.py#L67 ? There are other flags set by conda-build, so this seem like an arbitrary set to re-specify on the command line.

I think both[^1] of these should be removed to allow conda-build to specify everything needed for pip to work and keep the recipes as simple as possible. Or all the other flags should be added too.

Notice in https://github.com/conda-forge/packaging-feedstock/pull/32 where --no-build-isolation was being added to a recipe based on greyskull.

[^1]: Not completely sure PIP_NO_DEPENDENCIES is the same as --no-deps, as it seems to be it should be PIP_NO_DEPS. The translation is automatic, and there isn't a --no-dependencies flag. But I am sure that PIP_NO_BUILD_ISOLATION is correct. So at least that one could be trimmed here. Edit: tested and PIP_NO_DEPENDENCIES does work

henryiii avatar Nov 16 '24 06:11 henryiii

Notice in conda-forge/packaging-feedstock#32 where --no-build-isolation was being added to a recipe based on greyskull.

I've also added this to almost every Scikit-HEP package feedstock, and others, given grayskull's recommendations.

  • https://github.com/conda-forge/mplhep-feedstock/pull/87
  • https://github.com/conda-forge/cabinetry-feedstock/pull/5
  • https://github.com/conda-forge/pyhf-feedstock/pull/27
  • https://github.com/conda-forge/uproot-feedstock/pull/158
  • https://github.com/conda-forge/hatchling-feedstock/pull/59
  • https://github.com/conda-forge/staged-recipes/pull/27997
  • https://github.com/conda-forge/staged-recipes/pull/27414

matthewfeickert avatar Nov 16 '24 06:11 matthewfeickert

FYI, I tested PIP_NO_DEPENDENCIES and it does work.

henryiii avatar Nov 16 '24 06:11 henryiii

rattler-build does the same thing as conda-build

    // python variables
    // hard-code this because we never want pip's build isolation
    // https://github.com/conda/conda-build/pull/2972#discussion_r198290241
    //
    // Note that pip env "NO" variables are inverted logic.
    //    PIP_NO_BUILD_ISOLATION=False means don't use build isolation.
    insert!(vars, "PIP_NO_BUILD_ISOLATION", "False");
    // Some other env vars to have pip ignore dependencies. We supply them ourselves instead.
    insert!(vars, "PIP_NO_DEPENDENCIES", "True");
    insert!(vars, "PIP_IGNORE_INSTALLED", "True");

but their docs show using --no-deps --no-build-isolation as well. So cc @wolfv on if there is a reason I'm missing for having duplicated flags in the recipe files.

matthewfeickert avatar Nov 16 '24 07:11 matthewfeickert

Let's get rid of it! :) PRs always welcome

wolfv avatar Nov 16 '24 08:11 wolfv

Let's get rid of it! :) PRs always welcome

Thanks @wolfv. This is useful to know that I'm not missing anything. I'll PR things next week!

matthewfeickert avatar Nov 16 '24 15:11 matthewfeickert

So seems that we should revert grayskull's recommendation and also https://github.com/conda/conda-build/pull/4960.

matthewfeickert avatar Nov 17 '24 06:11 matthewfeickert

I think the only thing that's confusing me right now is that the comment says that the NO env vars are inverted, but then we do:

# we want _NO_ build isolation
PIP_NO_BUILD_ISOLATION=False
# we want _NO_ dependencies from pip
PIP_NO_DEPENDENCIES=True

Although I am also quite sure that I copied this from conda-build and that it has been working fine :)

wolfv avatar Nov 17 '24 08:11 wolfv

Too bad they didn't go with more clear PIP_BUILD_ISOLATION={disabled, enabled} :)

wolfv avatar Nov 17 '24 08:11 wolfv

https://github.com/pypa/pip/issues/5735

wolfv avatar Nov 17 '24 08:11 wolfv

These are automatically generated from the command line flags. Every command line flag in pip has an environment variable version, but that’s why they’re not customized.

henryiii avatar Nov 17 '24 17:11 henryiii

Conda-build sets PIP_NO_BUILD_ISOLATION and PIP_NO_DEPENDENCIES (See https://github.com/conda/conda-build/blob/0ad4bb6829b440caae9ae8a4db14ab1ff3788ab7/conda_build/build.py#L3020). Why is it also being set in every recipe in

We chose to be explicit in grayskull b/c conda-build has dropped those in the past in buggy releases and, those flags are not set in when using multiple outputs. I prefer to keep them explicit as they also inform users/recipe creators of what is happening under the hood.

ocefpaf avatar Nov 18 '24 09:11 ocefpaf

I prefer to keep them explicit as they also inform users/recipe creators of what is happening under the hood.

This is fine, but just to give some context (copying from https://github.com/conda-forge/staged-recipes/pull/28268#discussion_r1850849098 for findability by other people) I always assumed that the fact that grayskull had this recommendation of having --no-deps --no-build-isolation in the recipe meant that conda-forge wasn't using these flags anywhere as my assumption is that "If I have to specify something as the user then this means that it is my responsibility and that it isn't being dealt with elsewhere."

So I took away the opposite of the intended message.

matthewfeickert avatar Nov 20 '24 19:11 matthewfeickert

Personally, I think I'd assume that I could remove these flags and then depend on build isolation working - which is very much not the case. I'm not fond of do-nothing flags that can be removed without any affect. And the flags are somewhat arbitrary; personally if I were to put one in it would be --no-build-isolation, I don't think --no-deps is any more important than, say, --ignored-installed, which is not included).

If it doesn't work in multiple outputs, that's a bit of an issue - is this all flags, or just some?

henryiii avatar Nov 20 '24 19:11 henryiii