rattler-build icon indicating copy to clipboard operation
rattler-build copied to clipboard

bug: too many open files again and RATTLER_IO_CONCURRENCY_LIMIT env var not helping

Open beckermr opened this issue 7 months ago • 7 comments

I have hit this bug yet again. Interestingly, using the env var to limit IO concurrency ~or adjusting the ulimit~ does not help in this case. I expect that the env var should so this seems to be a bug.

cc @wolfv

command to run

RATTLER_IO_CONCURRENCY_LIMIT=8 CONDA_SUBDIR=osx-64 rattler-build build -m ../.ci_support/osx_64_python3.12.____cpython.yaml

for the recipe in this PR: https://github.com/conda-forge/stackvana-core-feedstock/pull/248

rattler-build output

% RATTLER_IO_CONCURRENCY_LIMIT=8 CONDA_SUBDIR=osx-64 rattler-build build -m ../.ci_support/osx_64_python3.12.____cpython.yaml

 ╭─ Finding outputs from recipe
 │ Loading variant config file: "../.ci_support/osx_64_python3.12.____cpython.yaml"
 │ Found 2 variants
 │ 
 │ Build variant: stackvana-core-impl-0.2025.19-py312h7628a71_1
 │ 
 │ ╭──────────────────────────┬────────────────────╮
 │ │ Variant                  ┆ Version            │
 │ ╞══════════════════════════╪════════════════════╡
 │ │ MACOSX_DEPLOYMENT_TARGET ┆ "10.13"            │
 │ │ channel_sources          ┆ "conda-forge"      │
 │ │ channel_targets          ┆ "conda-forge main" │
 │ │ python                   ┆ "3.12.* *_cpython" │
 │ │ target_platform          ┆ "osx-arm64"        │
 │ ╰──────────────────────────┴────────────────────╯
 │ 
 │ Build variant: stackvana-core-0.2025.19-py312h7e88756_1
 │ 
 │ ╭──────────────────────────┬─────────────────────────────╮
 │ │ Variant                  ┆ Version                     │
 │ ╞══════════════════════════╪═════════════════════════════╡
 │ │ MACOSX_DEPLOYMENT_TARGET ┆ "10.13"                     │
 │ │ channel_sources          ┆ "conda-forge"               │
 │ │ channel_targets          ┆ "conda-forge main"          │
 │ │ python                   ┆ "3.12.* *_cpython"          │
 │ │ stackvana_core_impl      ┆ "0.2025.19 py312h7628a71_1" │
 │ │ target_platform          ┆ "osx-arm64"                 │
 │ ╰──────────────────────────┴─────────────────────────────╯
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Running build for recipe: stackvana-core-impl-0.2025.19-py312h7628a71_1
 │
 │ ╭─ Fetching source code
 │ │ Fetching source from url: https://eups.lsst.codes/stack/src/tags/w_2025_19.list
 │ │ Validated SHA256 values of the downloaded file!
 │ │ Found valid source cache file.
 │ │ Copying source from url: /opt/conda-bld/src_cache/w_2025_19_a856b21e.list to /opt/conda-bld/bld/rattler-build_stackvana-core-impl_1746872596/work
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 │ ╭─ Resolving environments
 │ │ 
 │ │ Resolving host environment:
 │ │   Platform: osx-arm64 [__unix=0=0, __osx=13.7.5=0, __archspec=1=m2]
 │ │   Channels: 
 │ │    - file:///opt/conda-bld/
 │ │    - conda-forge
 │ │   Specs:
 │ │    - python 3.12.* *_cpython
 │ │    - rubin-env-nosysroot 10.*
 │ │ 
 │ │ ╭──────────────────────────────────────┬────────────────────┬──────────────────────────────┬─────────────┬────────────╮
 │ │ │ Package                              ┆ Version            ┆ Build                        ┆ Channel     ┆       Size │
 │ │ ╞══════════════════════════════════════╪════════════════════╪══════════════════════════════╪═════════════╪════════════╡
...
 │ │ 
 │ │ Finalized run dependencies (stackvana-core-impl-0.2025.19-py312h7628a71_1):
 │ │ ╭─────────────────────┬───────────────────────────────────────╮
 │ │ │ Name                ┆ Spec                                  │
 │ │ ╞═════════════════════╪═══════════════════════════════════════╡
 │ │ │ Run dependencies    ┆                                       │
 │ │ │ python              ┆                                       │
 │ │ │ rubin-env-nosysroot ┆ >=10.0.0,<11.0a0 (PC)                 │
 │ │ │ python_abi          ┆ 3.12.* *_cp312 (RE of [host: python]) │
 │ │ ╰─────────────────────┴───────────────────────────────────────╯
 │ │
 │ ╰─────────────────── (took 2 seconds)
 │ 
 │ Installing build environment
 │ ✔ Successfully updated the build environment
 │ 
 │ Installing host environment
 │
 ╰─────────────────── (took 2 seconds)
 │ ✔ validate cache       578 packages in 31ms
 │   installing packages  [00:00:00] [━━━━━━━━━━━━━━━━━━━━]   642/642 libopenvino-auto-plugin (+76)                                                                       Error:   × Failed to resolve dependencies: failed to fetch colorama-0.4.6-pyhd8ed1ab_1.conda
  ├─▶ failed to fetch colorama-0.4.6-pyhd8ed1ab_1.conda
  ├─▶ failed to open cache lock for reading: '/Users/beckermr/Library/Caches/rattler/cache/pkgs/colorama-0.4.6-pyhd8ed1ab_1.lock'
  ╰─▶ Too many open files (os error 24)

conda list

# packages in environment at /Users/beckermr/mambaforge/envs/cf-dev:
#
# Name                    Version                   Build  Channel
_python_abi3_support      1.0                  hd8ed1ab_2    conda-forge
admin-migrations          0.1                      pypi_0    pypi
adwaita-icon-theme        48.0                     unix_0    conda-forge
aiobotocore               2.22.0             pyhd8ed1ab_0    conda-forge
aiohappyeyeballs          2.6.1              pyhd8ed1ab_0    conda-forge
aiohttp                   3.11.18         py311h4921393_0    conda-forge
aioitertools              0.12.0             pyhd8ed1ab_1    conda-forge
aiosignal                 1.3.2              pyhd8ed1ab_0    conda-forge
anaconda-cli-base         0.5.2              pyhd8ed1ab_0    conda-forge
anaconda-client           1.13.0             pyh29332c3_1    conda-forge
annotated-types           0.7.0              pyhd8ed1ab_1    conda-forge
anyio                     4.9.0              pyh29332c3_0    conda-forge
appdirs                   1.4.4              pyhd8ed1ab_1    conda-forge
appnope                   0.1.4              pyhd8ed1ab_1    conda-forge
archspec                  0.2.5              pyhd8ed1ab_0    conda-forge
argcomplete               3.6.2              pyhd8ed1ab_0    conda-forge
arrow                     1.3.0              pyhd8ed1ab_1    conda-forge
astroid                   3.3.9           py311h267d04e_0    conda-forge
asttokens                 3.0.0              pyhd8ed1ab_1    conda-forge
async-timeout             5.0.1              pyhd8ed1ab_1    conda-forge
atk-1.0                   2.38.0               hd03087b_2    conda-forge
attrs                     25.3.0             pyh71513ae_0    conda-forge
aws-c-auth                0.9.0                hf8df148_6    conda-forge
aws-c-cal                 0.9.0                hd7db386_0    conda-forge
aws-c-common              0.12.2               h5505292_0    conda-forge
aws-c-compression         0.3.1                hc2321cf_4    conda-forge
aws-c-event-stream        0.5.4                h00dedb7_8    conda-forge
aws-c-http                0.10.1               h4486ab7_0    conda-forge
aws-c-io                  0.19.0               hff97710_1    conda-forge
aws-c-mqtt                0.13.0               hc33dfae_2    conda-forge
aws-c-s3                  0.7.17               h2b77d3b_1    conda-forge
aws-c-sdkutils            0.2.3                hc2321cf_4    conda-forge
aws-checksums             0.2.7                hc2321cf_0    conda-forge
aws-crt-cpp               0.32.5               he5a6918_2    conda-forge
aws-sdk-cpp               1.11.510             h22d0237_7    conda-forge
azure-core-cpp            1.14.0               hd50102c_0    conda-forge
azure-identity-cpp        1.10.0               hc602bab_0    conda-forge
azure-storage-blobs-cpp   12.13.0              h7585a09_1    conda-forge
azure-storage-common-cpp  12.8.0               h9ca1f76_1    conda-forge
azure-storage-files-datalake-cpp 12.12.0              hcdd55da_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backoff                   2.2.1              pyhd8ed1ab_1    conda-forge
backports                 1.0                pyhd8ed1ab_5    conda-forge
backports.functools_lru_cache 2.0.0              pyhd8ed1ab_1    conda-forge
backports.strenum         1.2.8              pyhd8ed1ab_0    conda-forge
backports.tarfile         1.2.0              pyhd8ed1ab_1    conda-forge
beautifulsoup4            4.13.4             pyha770c72_0    conda-forge
binaryornot               0.4.4              pyhd8ed1ab_2    conda-forge
black                     25.1.0          py311h267d04e_0    conda-forge
bleach                    6.2.0              pyh29332c3_4    conda-forge
blinker                   1.9.0              pyhff2d567_0    conda-forge
bokeh                     3.7.2              pyhd8ed1ab_1    conda-forge
boltons                   24.0.0             pyhd8ed1ab_1    conda-forge
boolean.py                5.0                pyhd8ed1ab_0    conda-forge
botocore                  1.37.3          pyge310_1234567_0    conda-forge
brotli                    1.1.0                hd74edd7_2    conda-forge
brotli-bin                1.1.0                hd74edd7_2    conda-forge
brotli-python             1.1.0           py311h3f08180_2    conda-forge
brotlipy                  0.7.0           py311h460d6c5_1007    conda-forge
bzip2                     1.0.8                h99b78c6_7    conda-forge
c-ares                    1.34.5               h5505292_0    conda-forge
ca-certificates           2025.4.26            hbd8a1cb_0    conda-forge
cachecontrol              0.14.3             pyha770c72_0    conda-forge
cachecontrol-with-filecache 0.14.3             pyhd8ed1ab_0    conda-forge
cachetools                5.5.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.4               h6a3b0d2_0    conda-forge
cctools                   1010.6               hb4fb6a3_6    conda-forge
cctools_osx-arm64         1010.6               h3b4f5d3_6    conda-forge
certifi                   2025.4.26          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py311h3a79f62_0    conda-forge
cfgv                      3.3.1              pyhd8ed1ab_1    conda-forge
chardet                   5.2.0              pyhd8ed1ab_3    conda-forge
charset-normalizer        3.4.2              pyhd8ed1ab_0    conda-forge
cirun                     0.30               pyhd8ed1ab_1    conda-forge
cleo                      2.1.0              pyhd8ed1ab_1    conda-forge
click                     8.1.8              pyh707e725_0    conda-forge
click-default-group       1.2.4              pyhd8ed1ab_1    conda-forge
click-log                 0.4.0              pyhd8ed1ab_1    conda-forge
click-option-group        0.5.6              pyhd8ed1ab_0    conda-forge
cloudpickle               3.1.1              pyhd8ed1ab_0    conda-forge
cmarkgfm                  2024.11.20      py311h917b07b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_1    conda-forge
comm                      0.2.2              pyhd8ed1ab_1    conda-forge
conda                     25.3.1          py311h267d04e_1    conda-forge
conda-build               25.4.2          py311h267d04e_0    conda-forge
conda-env                 2.6.0                         1    conda-forge
conda-forge-ci-setup      4.18.0          py311hd329421_100    conda-forge
conda-forge-feedstock-check-solvable 0.10.0             pyhd8ed1ab_0    conda-forge
conda-forge-feedstock-ops 0.12.0             pyhd8ed1ab_0    conda-forge
conda-forge-metadata      0.11.0             pyhd8ed1ab_1    conda-forge
conda-forge-pinning       2025.05.10.05.53.06      hd8ed1ab_0    conda-forge
conda-forge-tick          2025.5.29          pyhd8ed1ab_0    conda-forge
conda-forge-webservices   2025.4.12.dev9+g8a21169          pypi_0    pypi
conda-index               0.6.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     25.4.0             pyhd8ed1ab_0    conda-forge
conda-lock                3.0.1              pyh534df25_0    conda-forge
conda-oci                 0.1.dev2+gaa102fa          pypi_0    pypi
conda-oci-mirror          0.2.3              pyhd8ed1ab_0    conda-forge
conda-package-handling    2.4.0              pyh7900ff3_2    conda-forge
conda-package-streaming   0.11.0             pyhd8ed1ab_1    conda-forge
conda-recipe-manager      0.5.0              pyhd8ed1ab_0    conda-forge
conda-smithy              3.47.2.dev11+g6b6c9255          pypi_0    pypi
conda-souschef            2.2.3              pyhd8ed1ab_0    conda-forge
conda-suggest             0.1.1              pyh9f0ad1d_0    conda-forge
conda-suggest-conda-forge 2021.8.24            hf450f58_0    conda-forge
conda-verify              3.4.2              pyhd8ed1ab_1    conda-forge
contextlib2               21.6.0             pyhd8ed1ab_1    conda-forge
contourpy                 1.3.2           py311h210dab8_0    conda-forge
cookiecutter              2.6.0              pyhd8ed1ab_1    conda-forge
coverage                  7.8.0           py311h4921393_0    conda-forge
cpp-expected              1.1.0                h177bc72_1    conda-forge
cpython                   3.11.12         py311hd8ed1ab_0    conda-forge
cramjam                   2.10.0          py311h6bb77b2_0    conda-forge
crashtest                 0.4.1              pyhd8ed1ab_1    conda-forge
cryptography              44.0.3          py311h8be0713_0    conda-forge
curl                      8.13.0               h73640d1_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_1    conda-forge
cython                    3.1.0           py311h71112e5_0    conda-forge
cytoolz                   1.0.1           py311h917b07b_0    conda-forge
dask                      2025.4.1           pyhd8ed1ab_0    conda-forge
dask-core                 2025.4.1           pyhd8ed1ab_0    conda-forge
debugpy                   1.8.14          py311h155a34a_0    conda-forge
decorator                 5.2.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
depfinder                 2.10.0          py311h267d04e_2    conda-forge
deprecated                1.2.18             pyhd8ed1ab_0    conda-forge
dill                      0.4.0              pyhd8ed1ab_0    conda-forge
distlib                   0.3.9              pyhd8ed1ab_1    conda-forge
distributed               2025.4.1           pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_1    conda-forge
dnspython                 2.7.0              pyhff2d567_1    conda-forge
docutils                  0.21.2             pyhd8ed1ab_1    conda-forge
dotty-dict                1.3.1              pyhd8ed1ab_1    conda-forge
dulwich                   0.22.8          py311h3ff9189_0    conda-forge
editables                 0.5                pyhd8ed1ab_1    conda-forge
ensureconda               1.4.7              pyh29332c3_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_1    conda-forge
epoxy                     1.5.10               h1c322ee_1    conda-forge
evalidate                 2.0.5              pyhe01879c_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_1    conda-forge
execnet                   2.1.1              pyhd8ed1ab_1    conda-forge
executing                 2.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.7.0                h286801f_0    conda-forge
fastparquet               2024.11.0       py311h18599de_0    conda-forge
feedparser                6.0.11             pyhd8ed1ab_1    conda-forge
filelock                  3.18.0             pyhd8ed1ab_0    conda-forge
findpython                0.6.3              pyhff2d567_0    conda-forge
flake8                    7.2.0              pyhd8ed1ab_0    conda-forge
flaky                     3.8.1              pyhd8ed1ab_1    conda-forge
fmt                       11.1.4               h440487c_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_3    conda-forge
fontconfig                2.15.0               h1383a14_1    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.57.0          py311h4921393_0    conda-forge
freetype                  2.13.3               hce30654_1    conda-forge
fribidi                   1.0.10               h27ca646_0    conda-forge
frozendict                2.4.6           py311hae2e1ce_0    conda-forge
frozenlist                1.6.0           py311h87ddb2a_0    conda-forge
fsspec                    2025.3.2           pyhd8ed1ab_0    conda-forge
future                    1.0.0              pyhd8ed1ab_2    conda-forge
gdk-pixbuf                2.42.12              h7ddc832_0    conda-forge
gettext                   0.24.1               h3dcc1bd_0    conda-forge
gettext-tools             0.24.1               h493aca8_0    conda-forge
gflags                    2.2.2             hf9b8971_1005    conda-forge
git                       2.49.0          pl5321hd71a902_0    conda-forge
git-filter-repo           2.38.0             pyhd8ed1ab_1    conda-forge
gitdb                     4.0.12             pyhd8ed1ab_0    conda-forge
github3.py                4.0.1              pyhd8ed1ab_1    conda-forge
gitpython                 3.1.44             pyhff2d567_0    conda-forge
glib-tools                2.84.0               h1dc7a0c_0    conda-forge
glob2                     0.7                        py_0    conda-forge
glog                      0.7.1                heb240a5_0    conda-forge
gmp                       6.3.0                h7bae524_2    conda-forge
gql                       3.5.2              pyhd8ed1ab_0    conda-forge
graphite2                 1.3.13            hebf3989_1003    conda-forge
graphql-core              3.2.4              pyhd8ed1ab_0    conda-forge
graphviz                  12.2.1               hff64154_1    conda-forge
grayskull                 2.9.1              pyhd8ed1ab_0    conda-forge
gtk3                      3.24.43              h07173f4_5    conda-forge
gts                       0.7.6                he42f4ea_4    conda-forge
h11                       0.16.0             pyhd8ed1ab_0    conda-forge
h2                        4.2.0              pyhd8ed1ab_0    conda-forge
harfbuzz                  11.1.0               hab40de2_0    conda-forge
hatch                     1.14.1             pyhd8ed1ab_0    conda-forge
hatch-vcs                 0.4.0              pyhd8ed1ab_1    conda-forge
hatchling                 1.27.0           pypyhd8ed1ab_0    conda-forge
hicolor-icon-theme        0.17                 hce30654_2    conda-forge
hpack                     4.1.0              pyhd8ed1ab_0    conda-forge
httpcore                  1.0.9              pyh29332c3_0    conda-forge
httpx                     0.28.1             pyhd8ed1ab_0    conda-forge
hyperframe                6.1.0              pyhd8ed1ab_0    conda-forge
hyperlink                 21.0.0             pyh29332c3_1    conda-forge
icu                       75.1                 hfee45f7_0    conda-forge
id                        1.5.0              pyh29332c3_0    conda-forge
identify                  2.6.10             pyhd8ed1ab_0    conda-forge
idna                      3.10               pyhd8ed1ab_1    conda-forge
importlib-metadata        8.6.1              pyha770c72_0    conda-forge
importlib-resources       6.5.2              pyhd8ed1ab_0    conda-forge
importlib_metadata        8.6.1                hd8ed1ab_0    conda-forge
importlib_resources       6.5.2              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_1    conda-forge
inline-snapshot           0.23.0             pyh29332c3_0    conda-forge
intake                    2.0.8              pyhd8ed1ab_0    conda-forge
invoke                    2.2.0              pyhd8ed1ab_1    conda-forge
ipykernel                 6.29.5             pyh57ce528_0    conda-forge
ipython                   9.2.0              pyhfb0248b_0    conda-forge
ipython_pygments_lexers   1.1.1              pyhd8ed1ab_0    conda-forge
isodate                   0.7.2              pyhd8ed1ab_1    conda-forge
isort                     6.0.1              pyhd8ed1ab_1    conda-forge
jaraco.classes            3.4.0              pyhd8ed1ab_2    conda-forge
jaraco.context            6.0.1              pyhd8ed1ab_0    conda-forge
jaraco.functools          4.1.0              pyhd8ed1ab_0    conda-forge
jedi                      0.19.2             pyhd8ed1ab_1    conda-forge
jinja2                    3.1.6              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_1    conda-forge
joblib                    1.5.0              pyhd8ed1ab_0    conda-forge
jq                        1.7.1                h93a5062_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_1    conda-forge
jsonpointer               3.0.0           py311h267d04e_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_1    conda-forge
jsonschema-specifications 2025.4.1           pyh29332c3_0    conda-forge
jupyter_client            8.6.3              pyhd8ed1ab_1    conda-forge
jupyter_core              5.7.2              pyh31011fe_1    conda-forge
keyring                   25.6.0             pyh534df25_0    conda-forge
kiwisolver                1.4.7           py311h2c37856_0    conda-forge
krb5                      1.21.3               h237132a_0    conda-forge
lazyasd                   0.1.4                      py_1    conda-forge
lcms2                     2.17                 h7eeda09_0    conda-forge
ld64                      951.9                h4c6efb1_6    conda-forge
ld64_osx-arm64            951.9                hb6b49e2_6    conda-forge
lerc                      4.0.0                hd64df32_1    conda-forge
libabseil                 20250127.1      cxx17_h07bc746_0    conda-forge
libarchive                3.7.7                h3c2f2b0_4    conda-forge
libarrow                  20.0.0           h271b6f8_2_cpu    conda-forge
libarrow-acero            20.0.0           hf07054f_2_cpu    conda-forge
libarrow-dataset          20.0.0           hf07054f_2_cpu    conda-forge
libarrow-flight           20.0.0           h98dcbf4_2_cpu    conda-forge
libarrow-flight-sql       20.0.0           h98dcbf4_2_cpu    conda-forge
libarrow-gandiva          20.0.0           hfe7713d_2_cpu    conda-forge
libarrow-substrait        20.0.0           he749cb8_2_cpu    conda-forge
libasprintf               0.24.1               h493aca8_0    conda-forge
libasprintf-devel         0.24.1               h493aca8_0    conda-forge
libblas                   3.9.0           31_h10e41b3_openblas    conda-forge
libbrotlicommon           1.1.0                hd74edd7_2    conda-forge
libbrotlidec              1.1.0                hd74edd7_2    conda-forge
libbrotlienc              1.1.0                hd74edd7_2    conda-forge
libcblas                  3.9.0           31_hb3479ef_openblas    conda-forge
libcrc32c                 1.1.2                hbdafb3b_0    conda-forge
libcurl                   8.13.0               h73640d1_0    conda-forge
libcxx                    20.1.4               ha82da77_1    conda-forge
libdeflate                1.23                 hec38601_0    conda-forge
libedit                   3.1.20250104    pl5321hafb1f1b_0    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libevent                  2.1.12               h2757513_1    conda-forge
libexpat                  2.7.0                h286801f_0    conda-forge
libfabric                 2.1.0                hce30654_1    conda-forge
libfabric1                2.1.0                h5505292_1    conda-forge
libffi                    3.4.6                h1da3d7d_1    conda-forge
libfreetype               2.13.3               hce30654_1    conda-forge
libfreetype6              2.13.3               h1d14073_1    conda-forge
libgd                     2.3.3               hb2c3a21_11    conda-forge
libgettextpo              0.24.1               h493aca8_0    conda-forge
libgettextpo-devel        0.24.1               h493aca8_0    conda-forge
libgfortran               14.2.0             heb5dd2a_105    conda-forge
libgfortran5              14.2.0             h2c44a93_105    conda-forge
libgit2                   1.9.0                h30ef4c0_1    conda-forge
libglib                   2.84.0               hdff4504_0    conda-forge
libgoogle-cloud           2.36.0               h9484b08_1    conda-forge
libgoogle-cloud-storage   2.36.0               h7081f7f_1    conda-forge
libgrpc                   1.71.0               h857da87_1    conda-forge
libhwloc                  2.11.2          default_hbce5d74_1001    conda-forge
libiconv                  1.18                 hfe07756_1    conda-forge
libidn2                   2.3.8                h38aa460_0    conda-forge
libintl                   0.24.1               h493aca8_0    conda-forge
libintl-devel             0.24.1               h493aca8_0    conda-forge
libjpeg-turbo             3.1.0                h5505292_0    conda-forge
liblapack                 3.9.0           31_hc9a63f6_openblas    conda-forge
liblief                   0.16.4               h286801f_0    conda-forge
libllvm15                 15.0.7               h4429f82_5    conda-forge
libllvm16                 16.0.6               hc4b4ae8_4    conda-forge
libllvm17                 17.0.6               hc4b4ae8_3    conda-forge
libllvm18                 18.1.8               hc4b4ae8_3    conda-forge
liblzma                   5.8.1                h39f12f2_1    conda-forge
liblzma-devel             5.8.1                h39f12f2_1    conda-forge
libmamba                  2.1.1                h7c3736b_0    conda-forge
libmambapy                2.1.1           py311h3e4fba9_0    conda-forge
libnghttp2                1.64.0               h6d7220d_0    conda-forge
libopenblas               0.3.29          openmp_hf332438_0    conda-forge
libopentelemetry-cpp      1.20.0               h0181452_0    conda-forge
libopentelemetry-cpp-headers 1.20.0               hce30654_0    conda-forge
libparquet                20.0.0           h636d7b7_2_cpu    conda-forge
libpmix                   5.0.7                h6500a5a_0    conda-forge
libpng                    1.6.47               h3783ad8_0    conda-forge
libprotobuf               5.29.3               hccd9074_1    conda-forge
libre2-11                 2024.07.02           hd41c47c_3    conda-forge
librsvg                   2.58.4               h266df6f_3    conda-forge
libsodium                 1.0.20               h99b78c6_0    conda-forge
libsolv                   0.7.32               h13dfb9a_2    conda-forge
libsqlite                 3.49.2               h3f77e49_0    conda-forge
libssh2                   1.11.1               h9cc3647_0    conda-forge
libthrift                 0.21.0               h64651cc_0    conda-forge
libtiff                   4.7.0                h551f018_4    conda-forge
libunistring              0.9.10               h3422bc3_0    conda-forge
libutf8proc               2.10.0               hda25de7_0    conda-forge
libwebp-base              1.5.0                h2471fea_0    conda-forge
libxcb                    1.17.0               hdb1d25a_0    conda-forge
libxml2                   2.13.8               h52572c6_0    conda-forge
libxslt                   1.1.39               h223e5b9_0    conda-forge
libzlib                   1.3.1                h8359307_2    conda-forge
license-expression        30.4.1             pyhd8ed1ab_0    conda-forge
llvm-openmp               20.1.4               hdb05f8b_0    conda-forge
llvm-tools                18.1.8               hc4b4ae8_3    conda-forge
llvm-tools-18             18.1.8               hc4b4ae8_3    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lockfile                  0.12.2                     py_1    conda-forge
loky                      3.5.3              pyhd8ed1ab_0    conda-forge
lxml                      5.4.0           py311hfcb965d_0    conda-forge
lz4                       4.4.4           py311h3a49619_0    conda-forge
lz4-c                     1.10.0               h286801f_1    conda-forge
lzo                       2.10              h93a5062_1001    conda-forge
make                      4.4.1                hc9fafa5_2    conda-forge
mamba                     2.1.1                h105ca85_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_1    conda-forge
markupsafe                3.0.2           py311h4921393_1    conda-forge
matplotlib                3.10.3          py311ha1ab1f8_0    conda-forge
matplotlib-base           3.10.3          py311h031da69_0    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_1    conda-forge
matrix_client             0.4.0              pyhd8ed1ab_0    conda-forge
mbedtls                   3.5.1                h286801f_1    conda-forge
mccabe                    0.7.0              pyhd8ed1ab_1    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_1    conda-forge
menuinst                  2.2.0           py311h267d04e_0    conda-forge
more-itertools            10.7.0             pyhd8ed1ab_0    conda-forge
mpi                       1.0                     openmpi    conda-forge
msgpack-python            1.1.0           py311h2c37856_0    conda-forge
msrest                    0.6.21             pyh44b312d_0    conda-forge
multidict                 6.4.3           py311h30e7462_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
mypy                      1.15.0          py311h917b07b_0    conda-forge
mypy_extensions           1.1.0              pyha770c72_0    conda-forge
narwhals                  1.38.2             pyhe01879c_0    conda-forge
nbformat                  5.10.4             pyhd8ed1ab_1    conda-forge
ncurses                   6.5                  h5e97a16_3    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_1    conda-forge
networkx                  3.4.2              pyh267e887_2    conda-forge
nh3                       0.2.21           py39h52c9f89_1    conda-forge
nlohmann_json             3.11.3               h00cdb27_1    conda-forge
nodeenv                   1.9.1              pyhd8ed1ab_1    conda-forge
numpy                     2.2.5           py311h762c074_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_1    conda-forge
oniguruma                 6.9.10               h5505292_0    conda-forge
openjpeg                  2.5.3                h8a3d83b_0    conda-forge
openmpi                   5.0.7              h31ce4ef_100    conda-forge
openssl                   3.5.0                h81ee809_1    conda-forge
oras-py                   0.1.14             pyhd8ed1ab_0    conda-forge
orc                       2.1.2                hd90e43c_0    conda-forge
orjson                    3.10.18         py311h3ff9189_0    conda-forge
packaging                 25.0               pyh29332c3_1    conda-forge
pandas                    2.2.3           py311hca32420_3    conda-forge
pango                     1.56.3               h5fd7515_1    conda-forge
parameterized             0.9.0              pyhd8ed1ab_1    conda-forge
parso                     0.8.4              pyhd8ed1ab_1    conda-forge
partd                     1.4.2              pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h27ca646_1002    conda-forge
pathspec                  0.12.1             pyhd8ed1ab_1    conda-forge
pbs-installer             2025.4.9           pyhd8ed1ab_0    conda-forge
pcre2                     10.44                h297a79d_2    conda-forge
perl                      5.32.1          7_h4614cfb_perl5    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_1    conda-forge
pickleshare               0.7.5           pyhd8ed1ab_1004    conda-forge
pillow                    11.2.1          py311hb9ba9e9_0    conda-forge
pip                       25.1.1             pyh8b19718_0    conda-forge
pixi                      0.46.0               h4dcb070_0    conda-forge
pixman                    0.46.0               h2f9eb0b_0    conda-forge
pkginfo                   1.12.1.2           pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_2    conda-forge
platformdirs              4.3.8              pyhe01879c_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_1    conda-forge
poetry                    2.1.3              pyh534df25_0    conda-forge
poetry-core               2.1.3              pyhd8ed1ab_0    conda-forge
pre-commit                4.2.0              pyha770c72_0    conda-forge
progressbar2              4.5.0              pyhd8ed1ab_1    conda-forge
prometheus-cpp            1.3.0                h0967b3e_0    conda-forge
prompt-toolkit            3.0.51             pyha770c72_0    conda-forge
prompt_toolkit            3.0.51               hd8ed1ab_0    conda-forge
propcache                 0.3.1           py311h4921393_0    conda-forge
psutil                    7.0.0           py311h917b07b_0    conda-forge
pthread-stubs             0.4               hd74edd7_1002    conda-forge
ptyprocess                0.7.0              pyhd8ed1ab_1    conda-forge
pure_eval                 0.2.3              pyhd8ed1ab_1    conda-forge
py                        1.11.0             pyhd8ed1ab_1    conda-forge
py-lief                   0.16.4          py311h155a34a_0    conda-forge
py-rattler                0.9.0           py311h8be0713_0    conda-forge
pyarrow                   20.0.0          py311ha1ab1f8_0    conda-forge
pyarrow-core              20.0.0          py311he04fa90_0_cpu    conda-forge
pyarrow-hotfix            0.7                pyhd8ed1ab_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycodestyle               2.13.0             pyhd8ed1ab_0    conda-forge
pycosat                   0.6.6           py311h917b07b_2    conda-forge
pycparser                 2.22               pyh29332c3_1    conda-forge
pycryptodome              3.22.0          py311h5f4a806_0    conda-forge
pydantic                  2.11.4             pyh3cfb1c2_0    conda-forge
pydantic-core             2.33.2          py311hf245fc6_0    conda-forge
pydantic-extra-types      2.10.4             pyh3cfb1c2_0    conda-forge
pydantic-settings         2.9.1              pyh3cfb1c2_0    conda-forge
pyflakes                  3.3.2              pyhd8ed1ab_0    conda-forge
pygit2                    1.18.0          py311hd460543_0    conda-forge
pygithub                  2.6.1              pyhd8ed1ab_0    conda-forge
pygments                  2.19.1             pyhd8ed1ab_0    conda-forge
pygraphviz                1.14            py311hd3d57da_0    conda-forge
pyjwt                     2.10.1             pyhd8ed1ab_0    conda-forge
pylint                    3.3.7              pyhe01879c_0    conda-forge
pymongo                   4.11            py311h155a34a_0    conda-forge
pynacl                    1.5.0           py311h460d6c5_4    conda-forge
pynamodb                  6.0.2              pyhd8ed1ab_0    conda-forge
pyobjc-core               11.0            py311hab620ed_0    conda-forge
pyobjc-framework-cocoa    11.0            py311hab620ed_0    conda-forge
pyopenssl                 25.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.2.3              pyhd8ed1ab_1    conda-forge
pyperclip                 1.9.0              pyh534df25_3    conda-forge
pyproject_hooks           1.2.0              pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyha55dd90_7    conda-forge
pytest                    8.3.5              pyhd8ed1ab_0    conda-forge
pytest-cov                6.1.1              pyhd8ed1ab_0    conda-forge
pytest-forked             1.6.0              pyhd8ed1ab_1    conda-forge
pytest-mock               3.14.0             pyhd8ed1ab_1    conda-forge
pytest-rerunfailures      15.1               pyhd8ed1ab_0    conda-forge
pytest-xdist              3.6.1              pyhd8ed1ab_1    conda-forge
python                    3.11.9          h932a869_0_cpython    conda-forge
python-build              1.2.2.post1        pyhff2d567_1    conda-forge
python-dateutil           2.9.0.post0        pyhff2d567_1    conda-forge
python-dotenv             1.1.0              pyh29332c3_1    conda-forge
python-fastjsonschema     2.21.1             pyhd8ed1ab_0    conda-forge
python-gil                3.11.12              hd8ed1ab_0    conda-forge
python-gitlab             5.6.0              pyhff2d567_0    conda-forge
python-graphviz           0.20.3             pyh91182bf_2    conda-forge
python-installer          0.7.0              pyhff2d567_1    conda-forge
python-libarchive-c       5.2                pyh29332c3_0    conda-forge
python-rapidjson          1.20            py311h155a34a_1    conda-forge
python-semantic-release   9.21.1             pyh10f6f8f_0    conda-forge
python-slugify            8.0.4              pyhd8ed1ab_1    conda-forge
python-snappy             0.7.3              pyh4e55633_1    conda-forge
python-tzdata             2025.2             pyhd8ed1ab_0    conda-forge
python-utils              3.9.1              pyhff2d567_1    conda-forge
python_abi                3.11                    7_cp311    conda-forge
pytz                      2025.2             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2           py311h4921393_2    conda-forge
pyzmq                     26.4.0          py311h01f2145_0    conda-forge
qhull                     2020.2               h420ef59_5    conda-forge
rapidfuzz                 3.13.0          py311h155a34a_0    conda-forge
rattler-build             0.41.0               h8dba533_0    conda-forge
rattler-build-conda-compat 1.4.3              pyhd8ed1ab_0    conda-forge
re2                       2024.07.02           h6589ca4_3    conda-forge
readchar                  4.2.0              pyhd8ed1ab_1    conda-forge
readline                  8.2                  h1d1bf99_2    conda-forge
readme_renderer           44.0               pyhd8ed1ab_1    conda-forge
referencing               0.36.2             pyh29332c3_0    conda-forge
reproc                    14.2.5.post0         h5505292_0    conda-forge
reproc-cpp                14.2.5.post0         h286801f_0    conda-forge
requests                  2.32.3             pyhd8ed1ab_1    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_1    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_1    conda-forge
rever                     0.5.1              pyhd8ed1ab_1    conda-forge
rfc3986                   2.0.0              pyhd8ed1ab_1    conda-forge
rich                      14.0.0             pyh29332c3_0    conda-forge
ripgrep                   14.1.1               h0ef69ab_1    conda-forge
rpds-py                   0.24.0          py311hc9d6b66_0    conda-forge
ruamel.yaml               0.18.10         py311h917b07b_0    conda-forge
ruamel.yaml.clib          0.2.8           py311hae2e1ce_1    conda-forge
ruamel.yaml.jinja2        0.2.7              pyhd8ed1ab_1    conda-forge
ruamel_yaml               0.15.80         py311heffc1b2_1009    conda-forge
ruff                      0.11.9          py311h2ed1275_0    conda-forge
s3fs                      2025.3.2           pyhd8ed1ab_0    conda-forge
scipy                     1.15.2          py311h0675101_0    conda-forge
scrypt                    0.8.27          py311hf787bfc_0    conda-forge
semver                    3.0.4              pyhd8ed1ab_0    conda-forge
setproctitle              1.3.6           py311h917b07b_0    conda-forge
setuptools                80.1.0             pyhff2d567_0    conda-forge
setuptools-scm            8.3.1              pyhd8ed1ab_0    conda-forge
setuptools_scm            8.3.1                hd8ed1ab_0    conda-forge
sgmllib3k                 1.0.0              pyhd8ed1ab_1    conda-forge
shellcheck                0.10.0               hecfb573_0    conda-forge
shellingham               1.5.4              pyhd8ed1ab_1    conda-forge
shyaml                    0.6.2              pyhd3deb0d_0    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
simdjson                  3.12.3               ha393de7_0    conda-forge
six                       1.17.0             pyhd8ed1ab_0    conda-forge
smmap                     5.0.2              pyhd8ed1ab_0    conda-forge
snappy                    1.2.1                h98b9ce2_1    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_1    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_1    conda-forge
soupsieve                 2.7                pyhd8ed1ab_0    conda-forge
stack_data                0.6.3              pyhd8ed1ab_1    conda-forge
stdlib-list               0.11.1             pyhd8ed1ab_0    conda-forge
stopit                    1.1.2              pyhd8ed1ab_1    conda-forge
streamz                   0.6.4              pyhd8ed1ab_1    conda-forge
tapi                      1300.6.5             h03f4b80_0    conda-forge
tar                       1.34                 h7cb298e_1    conda-forge
tblib                     3.1.0              pyhd8ed1ab_0    conda-forge
text-unidecode            1.3                pyhd8ed1ab_2    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_1    conda-forge
tomli                     2.2.1              pyhd8ed1ab_1    conda-forge
tomli-w                   1.2.0              pyhd8ed1ab_0    conda-forge
tomlkit                   0.13.2             pyha770c72_1    conda-forge
toolz                     1.0.0              pyhd8ed1ab_1    conda-forge
tornado                   6.4.2           py311h917b07b_0    conda-forge
tqdm                      4.67.1             pyhd8ed1ab_1    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_1    conda-forge
trove-classifiers         2025.5.9.12        pyhd8ed1ab_0    conda-forge
truststore                0.10.1             pyh29332c3_0    conda-forge
twine                     6.1.0              pyh29332c3_0    conda-forge
typer                     0.15.3             pyhf21524f_0    conda-forge
typer-slim                0.15.3             pyh29332c3_0    conda-forge
typer-slim-standard       0.15.3               h1a15894_0    conda-forge
types-jsonschema          4.23.0.20241208    pyhd8ed1ab_0    conda-forge
types-python-dateutil     2.9.0.20241206     pyhd8ed1ab_0    conda-forge
types-pyyaml              6.0.12.20250402    pyhd8ed1ab_0    conda-forge
types-requests            2.32.0.20250306    pyhd8ed1ab_0    conda-forge
typing-extensions         4.13.2               h0e9735f_0    conda-forge
typing-inspection         0.4.0              pyhd8ed1ab_0    conda-forge
typing_extensions         4.13.2             pyh29332c3_0    conda-forge
tzdata                    2025b                h78e105d_0    conda-forge
ukkonen                   1.0.1           py311h2c37856_5    conda-forge
unicodedata2              16.0.0          py311h917b07b_0    conda-forge
uritemplate               4.1.1              pyhd8ed1ab_1    conda-forge
urllib3                   2.4.0              pyhd8ed1ab_0    conda-forge
userpath                  1.9.2              pyhd8ed1ab_0    conda-forge
uv                        0.7.3                hb4c02be_0    conda-forge
virtualenv                20.31.2            pyhd8ed1ab_0    conda-forge
vsts-python-api           0.1.25             pyhd8ed1ab_2    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_1    conda-forge
webencodings              0.5.1              pyhd8ed1ab_3    conda-forge
wget                      1.21.4               he2df1f1_0    conda-forge
wheel                     0.45.1             pyhd8ed1ab_1    conda-forge
wrapt                     1.17.2          py311h917b07b_0    conda-forge
wurlitzer                 3.1.1              pyhd8ed1ab_1    conda-forge
xattr                     1.1.0           py311h460d6c5_1    conda-forge
xdoctest                  1.2.0              pyhd8ed1ab_0    conda-forge
xmltodict                 0.14.2             pyhd8ed1ab_1    conda-forge
xonsh                     0.19.4          py311h267d04e_0    conda-forge
xorg-libxau               1.0.12               h5505292_0    conda-forge
xorg-libxdmcp             1.1.5                hd74edd7_0    conda-forge
xyzservices               2025.4.0           pyhd8ed1ab_0    conda-forge
xz                        5.8.1                h9a6d368_1    conda-forge
xz-gpl-tools              5.8.1                h9a6d368_1    conda-forge
xz-tools                  5.8.1                h39f12f2_1    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
yaml-cpp                  0.8.0                h13dd4ca_0    conda-forge
yamllint                  1.37.1             pyhe01879c_0    conda-forge
yarl                      1.20.0          py311h4921393_0    conda-forge
yq                        3.4.3              pyhd8ed1ab_1    conda-forge
zeromq                    4.3.5                hc1bb282_7    conda-forge
zict                      3.0.0              pyhd8ed1ab_1    conda-forge
zipp                      3.21.0             pyhd8ed1ab_1    conda-forge
zlib                      1.3.1                h8359307_2    conda-forge
zstandard                 0.23.0          py311h917b07b_2    conda-forge
zstd                      1.5.7                h6491c7d_2    conda-forge

beckermr avatar May 10 '25 10:05 beckermr

For reference, the latest rattler-build release on osx-64 was pulled because we ran into a problem with HTTP requests / rust 1.86.0 / osx-64. I am not sure if that's why this is not working.

wolfv avatar May 10 '25 12:05 wolfv

Ah would this effect osx-arm64 too? We might want to pull that one too?

beckermr avatar May 10 '25 13:05 beckermr

Nope, did not affect osx-arm64 :)

wolfv avatar May 10 '25 13:05 wolfv

Hmm, this was part of the 0.40.0 release and thus should have worked.

@beckermr was this when building locally (on osx-arm64 for osx-64, I suppose?)?

@iamthebot did you all already test this feature?

Of course, one workaround is to change the ulimit on the system (as many package managers require).

Something like:

sudo launchctl limit maxfiles 10240 unlimited

wolfv avatar May 12 '25 08:05 wolfv

@beckermr was this when building locally (on osx-arm64 for osx-64, I suppose?)?

yes

beckermr avatar May 12 '25 11:05 beckermr

@wolfv weird, I haven't hit this at all since the 0.40.0 release. Entirely possible I missed something in the PR though... but nothing is jumping out at me looking at the code. None of our other developers using mac machines are running into it either (though even before, it was somewhat random).

iamthebot avatar May 13 '25 00:05 iamthebot

@beckermr I will have some time wednesday to try and repro this + do a bit of debugging.

iamthebot avatar May 13 '25 00:05 iamthebot