sci icon indicating copy to clipboard operation
sci copied to clipboard

dev-python/snakemake: version bump

Open mschubert opened this issue 10 months ago • 6 comments

mschubert avatar Feb 17 '25 18:02 mschubert

Fails on my end with:

assets.AssetDownloadError: Failed to download asset https://raw.githubusercontent.com/snakemake/snakemake/main/LICENSE.md: <urlopen error [Errno -2] Name or service not known>

It seems that setup.py downloads a range of assets, which will fail if there is no internet connection during the installation.

~~I'm not sure if/how this is fixable, I'll check the pypi tarball.~~

We should be able to fix that by providing all the missing files as additional sources.

Is there an example of another ebuild that already does this?

mschubert avatar Feb 24 '25 20:02 mschubert

Is there an example of another ebuild that already does this?

The usual solution is to use the pypi release tarball instead of the github tag tarball. But not sure if that will resolve the problem in this case.

Nowa-Ammerlaan avatar Feb 26 '25 13:02 Nowa-Ammerlaan

I can confirm that the pypi tarball does not include the assets; I'm open to suggestions what a good solution is here.

mschubert avatar Feb 26 '25 18:02 mschubert

I can confirm that the pypi tarball does not include the assets; I'm open to suggestions what a good solution is here.

Perhaps we should ask upstream to provide for us a tarball that we can use to build without requiring an internet connection.

Nowa-Ammerlaan avatar Mar 24 '25 12:03 Nowa-Ammerlaan

I can definitely raise that, but given that they have 1.1k open issues it might take a while.

How strict is the "no internet" rule for sci, do we have to fix this first or can I put it on my todo for a follow-up PR?

mschubert avatar Mar 26 '25 11:03 mschubert

How strict is the "no internet" rule for sci, do we have to fix this first or can I put it on my todo for a follow-up PR?

It's strict, because it cannot build otherwise (i.e. I get the failure I shared before). In the current state this ebuild is not usable at all, so I don't think it makes sense to merge it now.

Since we did not have this problem with earlier versions of snakemake, maybe we can find the exact point where this broke and add the version directly before that.

Nowa-Ammerlaan avatar Mar 26 '25 12:03 Nowa-Ammerlaan

@Nowa-Ammerlaan as far as I have seen, the new version now includes the "Assets" on PyPI

mschubert avatar Jul 10 '25 13:07 mschubert

@Nowa-Ammerlaan as far as I have seen, the new version now includes the "Assets" on PyPI

Still failing on my end:

>>> Emerging (35 of 35) dev-python/snakemake-9.7.1::localrepo
* snakemake-9.7.1.tar.gz BLAKE2B SHA512 size ;-) ...                                                                                                                                                                  [ ok ]
>>> Unpacking source...
>>> Unpacking snakemake-9.7.1.tar.gz to /tmp/portage/dev-python/snakemake-9.7.1/work
>>> Source unpacked in /tmp/portage/dev-python/snakemake-9.7.1/work
>>> Preparing source in /tmp/portage/dev-python/snakemake-9.7.1/work/snakemake-9.7.1 ...
* Build system packages:
*   dev-python/gpep517            : 19
*   dev-python/installer          : 0.7.0
*   dev-python/setuptools         : 80.9.0-r1
*   dev-python/setuptools-rust    : 1.11.1
*   dev-python/setuptools-scm     : 8.3.1
*   dev-python/wheel              : 0.45.1
>>> Source prepared.
>>> Configuring source in /tmp/portage/dev-python/snakemake-9.7.1/work/snakemake-9.7.1 ...
>>> Source configured.
>>> Compiling source in /tmp/portage/dev-python/snakemake-9.7.1/work/snakemake-9.7.1 ...
* python3_12: running distutils-r1_run_phase distutils-r1_python_compile
*   Building the wheel for snakemake-9.7.1 via setuptools.build_meta
python3.12 -m gpep517 build-wheel --prefix=/usr --backend setuptools.build_meta --output-fd 3 --wheel-dir /tmp/portage/dev-python/snakemake-9.7.1/work/snakemake-9.7.1-python3_12/wheel
2025-07-13 11:59:14,263 gpep517 INFO Building wheel via backend setuptools.build_meta
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/usr/lib/python3.12/site-packages/gpep517/__main__.py", line 242, in <module>
sys.exit(main())
^^^^^^
File "/usr/lib/python3.12/site-packages/gpep517/__main__.py", line 238, in main
return func(args)
^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/gpep517/__main__.py", line 45, in build_wheel
print(build_wheel_impl(args, args.wheel_dir), file=out)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/gpep517/build.py", line 180, in build_wheel_impl
wheel_name = backend.build_wheel(str(wheel_dir), args.config_json)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 432, in build_wheel
return _build(['bdist_wheel'])
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 423, in _build
return self._build_with_temp_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
self.run_setup()
File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 317, in run_setup
exec(code, locals())
File "<string>", line 19, in <module>
File "/tmp/portage/dev-python/snakemake-9.7.1/work/snakemake-9.7.1/src/snakemake/assets/__init__.py", line 551, in deploy
fout.write(asset.get_content())
^^^^^^^^^^^^^^^^^^^
File "/tmp/portage/dev-python/snakemake-9.7.1/work/snakemake-9.7.1/src/snakemake/assets/__init__.py", line 46, in get_content
raise err
assets.AssetDownloadError: Failed to download asset https://cdn.tailwindcss.com/[email protected],[email protected]: <urlopen error [Errno -2] Name or service not known>
* ERROR: dev-python/snakemake-9.7.1::localrepo failed (compile phase):

Nowa-Ammerlaan avatar Jul 13 '25 10:07 Nowa-Ammerlaan

Hm, I need a way to reliably test this myself and not bother you with all the time time :laughing:

(I can't disconnect the network on my prefix)

mschubert avatar Jul 16 '25 08:07 mschubert

Hm, I need a way to reliably test this myself and not bother you with all the time time 😆

(I can't disconnect the network on my prefix)

This should work to reproduce: FEATURES="network-sandbox" emerge -1 snakemake.

There is no need to actually disconnect anything, portage should isolate network connectivity for just that emerge.

Nowa-Ammerlaan avatar Jul 16 '25 10:07 Nowa-Ammerlaan

Great, thanks!

But this either does not work on prefix, or I can't reproduce the fetch.

I'm also confused because snakemake should not fetch the assets that are already in the PyPI tarball.

mschubert avatar Jul 16 '25 13:07 mschubert

But this either does not work on prefix, or I can't reproduce the fetch.

We can determine which is the case by putting an explicit wget or curl with || die in the prepare phase, if it does not die then the network sandbox is not working.

Nowa-Ammerlaan avatar Jul 18 '25 08:07 Nowa-Ammerlaan

I can confirm that FEATURES=network-sandbox does not work on prefix. I'll see if I can set up a VM to properly test this.

mschubert avatar Jul 21 '25 12:07 mschubert

I now added a patch to fix the asset download; this works on my VM

mschubert avatar Jul 26 '25 16:07 mschubert

Thanks :+1:

Nowa-Ammerlaan avatar Aug 02 '25 11:08 Nowa-Ammerlaan