admin-requests icon indicating copy to clipboard operation
admin-requests copied to clipboard

Published 20 hours ago • verified publisher icon conda-forge

Create checksum_mismatch.yml

Open dward4 opened this issue 1 year ago • 4 comments

Both of these packages have a checksum mistmach and do not download and install.

Guidelines for marking packages as broken:

  • We prefer to patch the repo data (see here) instead of marking packages as broken. This alternative workflow makes environments more reproducible.
  • Packages with requirements/metadata that are too strict but otherwise work are not technically broken and should not be marked as such.
  • Packages with missing metadata can be marked as broken on a temporary basis but should be patched in the repo data and be marked unbroken later.
  • In some cases where the number of users of a package is small or it is used by the maintainers only, we can allow packages to be marked broken more liberally.
  • We (conda-forge/core) try to make a decision on these requests within 24 hours.

What will happen when a package is marked broken?

  • Our bots will add the broken label to the package. The main label will remain on the package and this is normal.
  • Our bots will rebuild our repodata patches to remove this package from the repodata.
  • In a few hours after the anaconda.org CDN picks up the new patches, you will no longer be able to install the package from the main channel.

Checklist:

  • [x] I want to mark a package as broken (or not broken):

    • [x] Added a description of the problem with the package in the PR description.
    • [ ] Pinged the team for the package for their input.

  • [ ] I want to archive a feedstock:

    • [ ] Pinged the team for that feedstock for their input.
    • [ ] Make sure you have opened an issue on the feedstock explaining why it was archived.
    • [ ] Linked that issue in this PR description.
    • [ ] Added links to any other relevant issues/PRs in the PR description.

  • [ ] I want to request (or revoke) access to an opt-in CI resource:

    • [ ] Pinged the relevant feedstock team(s)
    • [ ] Added a small description explaining why access is needed

dward4 avatar Jan 18 '24 19:01 dward4

We need to report this to anaconda inc maybe as well?

beckermr avatar Jan 18 '24 19:01 beckermr

Both of these packages have a checksum mistmach and do not download and install.

Works for me. All checksums I can think of match and installation works:

# curl -sLO 'https://conda.anaconda.org/conda-forge/win-64/xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2'
# curl -sLO 'https://conda.anaconda.org/conda-forge/win-64/pcre2-10.40-h17e33f8_0.tar.bz2'
# sha256sum *.tar.bz2
5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57  pcre2-10.40-h17e33f8_0.tar.bz2
34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11  xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2
# md5sum *.tar.bz2
2519de0d9620dc2bc7e19caf6867136d  pcre2-10.40-h17e33f8_0.tar.bz2
3eb2af45e531888ead3d80f85ff825df  xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2
# curl -sL 'https://conda.anaconda.org/conda-forge/win-64/repodata.json' > repodata.json
# <repodata.json jq -r '.["packages"]["pcre2-10.40-h17e33f8_0.tar.bz2"]|[.sha256, .md5]'
[
  "5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57",
  "2519de0d9620dc2bc7e19caf6867136d"
]
# <repodata.json jq -r '.["packages"]["xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2"]|[.sha256, .md5]'
[
  "34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11",
  "3eb2af45e531888ead3d80f85ff825df"
]
# curl -sL 'https://conda.anaconda.org/conda-forge/win-64/repodata.json.bz2' | bzip2 -d > repodata.json
# <repodata.json jq -r '.["packages"]["pcre2-10.40-h17e33f8_0.tar.bz2"]|[.sha256, .md5]'
[
  "5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57",
  "2519de0d9620dc2bc7e19caf6867136d"
]
# <repodata.json jq -r '.["packages"]["xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2"]|[.sha256, .md5]'
[
  "34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11",
  "3eb2af45e531888ead3d80f85ff825df"
]
# curl -sL 'https://conda.anaconda.org/conda-forge/win-64/repodata.json.zst' | zstd -d > repodata.json
# <repodata.json jq -r '.["packages"]["pcre2-10.40-h17e33f8_0.tar.bz2"]|[.sha256, .md5]'
[
  "5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57",
  "2519de0d9620dc2bc7e19caf6867136d"
]
# <repodata.json jq -r '.["packages"]["xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2"]|[.sha256, .md5]'
[
  "34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11",
  "3eb2af45e531888ead3d80f85ff825df"
]
# mkdir pcre2-10.40-h17e33f8_0
# bsdtar -xC pcre2-10.40-h17e33f8_0 -f pcre2-10.40-h17e33f8_0.tar.bz2
# ( cd pcre2-10.40-h17e33f8_0 && <info/paths.json jq -r '.paths[]|[._path, .sha256]|join("\n")' | xargs -n2 -- sh -c 'sha256sum "${1}" | grep -qF "${2}" || echo check failed for "${1}"' -- )
# mkdir xorg-libxdmcp-1.1.3-h301d43c_0
# bsdtar -xC xorg-libxdmcp-1.1.3-h301d43c_0 -f xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2 
# ( cd xorg-libxdmcp-1.1.3-h301d43c_0 && <info/paths.json jq -r '.paths[]|[._path, .sha256]|join("\n")' | xargs -n2 -- sh -c 'sha256sum "${1}" | grep -qF "${2}" || echo check failed for "${1}"' -- )
# CONDA_SUBDIR=win-64 conda create -qynp pcre2=10.40=h17e33f8_0
Channels:
 - conda-forge
Platform: win-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/envs/p

  added / updated specs:
    - pcre2==10.40=h17e33f8_0


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    bzip2-1.0.8                |       hcfcfb64_5         122 KB  conda-forge
    libzlib-1.2.13             |       hcfcfb64_5          54 KB  conda-forge
    pcre2-10.40                |       h17e33f8_0         1.9 MB  conda-forge
    ucrt-10.0.22621.0          |       h57928b3_0         1.2 MB  conda-forge
    vc-14.3                    |      hcf57466_18          17 KB  conda-forge
    vc14_runtime-14.38.33130   |      h82b7239_18         732 KB  conda-forge
    vs2015_runtime-14.38.33130 |      hcb4865c_18          17 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         4.1 MB

The following NEW packages will be INSTALLED:

  bzip2              conda-forge/win-64::bzip2-1.0.8-hcfcfb64_5 
  libzlib            conda-forge/win-64::libzlib-1.2.13-hcfcfb64_5 
  pcre2              conda-forge/win-64::pcre2-10.40-h17e33f8_0 
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0 
  vc                 conda-forge/win-64::vc-14.3-hcf57466_18 
  vc14_runtime       conda-forge/win-64::vc14_runtime-14.38.33130-h82b7239_18 
  vs2015_runtime     conda-forge/win-64::vs2015_runtime-14.38.33130-hcb4865c_18 


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate p
#
# To deactivate an active environment, use
#
#     $ conda deactivate

# CONDA_SUBDIR=win-64 conda create -qynx xorg-libxdmcp=1.1.3=h301d43c_0
Channels:
 - conda-forge
Platform: win-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/envs/x

  added / updated specs:
    - xorg-libxdmcp==1.1.3=h301d43c_0


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    m2w64-gcc-libgfortran-5.3.0|                6         342 KB  conda-forge
    m2w64-gcc-libs-5.3.0       |                7         520 KB  conda-forge
    m2w64-gcc-libs-core-5.3.0  |                7         214 KB  conda-forge
    m2w64-gmp-6.1.0            |                2         726 KB  conda-forge
    m2w64-libwinpthread-git-5.0.0.4634.697f757|                2          31 KB  conda-forge
    msys2-conda-epoch-20160418 |                1           3 KB  conda-forge
    xorg-libxdmcp-1.1.3        |       h301d43c_0          65 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         1.9 MB

The following NEW packages will be INSTALLED:

  m2w64-gcc-libgfor~ conda-forge/win-64::m2w64-gcc-libgfortran-5.3.0-6 
  m2w64-gcc-libs     conda-forge/win-64::m2w64-gcc-libs-5.3.0-7 
  m2w64-gcc-libs-co~ conda-forge/win-64::m2w64-gcc-libs-core-5.3.0-7 
  m2w64-gmp          conda-forge/win-64::m2w64-gmp-6.1.0-2 
  m2w64-libwinpthre~ conda-forge/win-64::m2w64-libwinpthread-git-5.0.0.4634.697f757-2 
  msys2-conda-epoch  conda-forge/win-64::msys2-conda-epoch-20160418-1 
  xorg-libxdmcp      conda-forge/win-64::xorg-libxdmcp-1.1.3-h301d43c_0 


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate x
#
# To deactivate an active environment, use
#
#     $ conda deactivate

mbargull avatar Jan 18 '24 19:01 mbargull

I'm seeing the expected checksum now as well via certiutil:

certutil -hashfile .\win-64_pcre2-10.40-h17e33f8_0.tar.bz2 SHA256 SHA256 hash of .\win-64_pcre2-10.40-h17e33f8_0.tar.bz2: 5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57

I tried a fresh install of miniconda, made a new environment and i'm still getting errors:

ChecksumMismatchError: Conda detected a mismatch between the expected content and downloaded content for url 'https://conda.anaconda.org/conda-forge/win-64/pcre2-10.40-h17e33f8_0.tar.bz2'. download saved to: C:\Users<SSO>\AppData\Local\miniconda3\pkgs\pcre2-10.40-h17e33f8_0.tar.bz2 expected sha256: 5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57 actual sha256: 2816cfa047dc46376c52dd5a33fa526b23703c9aea1e6ca6e7f4c77d00dc0b40

ChecksumMismatchError: Conda detected a mismatch between the expected content and downloaded content for url 'https://conda.anaconda.org/conda-forge/win-64/pcre2-10.40-h17e33f8_0.tar.bz2'. download saved to: C:\Users<SSO>\AppData\Local\miniconda3\pkgs\pcre2-10.40-h17e33f8_0.tar.bz2 expected sha256: 5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57 actual sha256: 2816cfa047dc46376c52dd5a33fa526b23703c9aea1e6ca6e7f4c77d00dc0b40

Do you have guidance on where to go next? I couldn't find how conda executes checksums. Thank you for all your help!

dward4 avatar Jan 23 '24 20:01 dward4

Do you have guidance on where to go next? I couldn't find how conda executes checksums. Thank you for all your help!

It's at https://github.com/conda/conda/blob/545e3c82222cf4b774eadd64c7c2a4a6b25066a1/conda/gateways/disk/read.py#L71, essentially delegating to the Python stdlib.

jaimergp avatar Aug 21 '24 17:08 jaimergp