mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz cannot be downloaded

[dalboris]

I'm trying to install harfbuzz with vcpkg version 2020.11-1 on Windows 7. The reason I'm using version 2020.11-1 is that any later version comes with Python 3.9+ which isn't supported on Windows 7.

Part of the dependencies to install harfbuzz is the package brotli which attempts to download mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz but fails. Indeed, the file seems to not exist on msys2 neither any of its mirrors.

Would it be possible to add back this file to the msys2 packages, so that using vcpkg on Windows 7 is somewhat possible? Thank you!

More info: https://github.com/harfbuzz/harfbuzz/issues/3421

vcpkg install output:

$ git clone https://github.com/Microsoft/vcpkg.git
$ cd vcpkg
$ git checkout 2020.11-1
$ .\bootstrap-vcpkg.bat
$ .\vcpkg install freetype:x64-windows harfbuzz:x64-windows
Computing installation plan...
The following packages will be built and installed:
  * brotli[core]:x64-windows
  * bzip2[core]:x64-windows
    freetype[bzip2,core,png]:x64-windows
    harfbuzz[core]:x64-windows
  * libpng[core]:x64-windows
  * ragel[core]:x64-windows
  * zlib[core]:x64-windows
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Starting package 1/7: brotli:x64-windows
Building package brotli[core]:x64-windows...
Could not locate cached archive: C:\Users\Boris\AppData\Local\vcpkg\archives\d1\d15b61925bc1d3595878690a0e189ce8a37e6849.zip
-- Using cached C:/Users/Boris/vcpkg/downloads/google-brotli-e61745a6b7add50d380cfd7d3883dd6c62fc2c71.tar.gz
-- Cleaning sources at C:/Users/Boris/vcpkg/buildtrees/brotli/src/6c62fc2c71-a8c4ea9278.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/Users/Boris/vcpkg/downloads/google-brotli-e61745a6b7add50d380cfd7d3883dd6c62fc2c71.tar.gz
-- Applying patch install.patch
-- Applying patch fix-arm-uwp.patch
-- Applying patch pkgconfig.patch
-- Using source at C:/Users/Boris/vcpkg/buildtrees/brotli/src/6c62fc2c71-a8c4ea9278.clean
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Downloading https://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 22;"HTTP response code said error"
-- Downloading https://www2.futureware.at/~nickoe/msys2-mirror/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://www2.futureware.at/~nickoe/msys2-mirror/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 22;"HTTP response code said error"
-- Downloading https://mirror.yandex.ru/mirrors/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://mirror.yandex.ru/mirrors/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 22;"HTTP response code said error"
-- Downloading https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 22;"HTTP response code said error"
-- Downloading https://mirrors.ustc.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://mirrors.ustc.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 22;"HTTP response code said error"
-- Downloading https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 6;"Couldn't resolve host name"
-- Downloading https://mirror.selfnet.de/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://mirror.selfnet.de/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 35;"SSL connect error"
-- Downloading https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz...
-- Downloading https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz... Failed. Status: 35;"SSL connect error"
CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:182 (message):

      Failed to download file.
      If you use a proxy, please set the HTTPS_PROXY and HTTP_PROXY environment
      variables to "https://user:password@your-proxy-ip-address:port/".
      Otherwise, please submit an issue at https://github.com/Microsoft/vcpkg/issues

Call Stack (most recent call first):
  scripts/cmake/vcpkg_acquire_msys.cmake:88 (vcpkg_download_distfile)
  scripts/cmake/vcpkg_acquire_msys.cmake:136 (msys_package_download)
  scripts/cmake/vcpkg_find_acquire_program.cmake:427 (vcpkg_acquire_msys)
  scripts/cmake/vcpkg_fixup_pkgconfig.cmake:271 (vcpkg_find_acquire_program)
  ports/brotli/portfile.cmake:25 (vcpkg_fixup_pkgconfig)
  scripts/ports.cmake:135 (include)


Error: Building package brotli:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: brotli:x64-windows
  Vcpkg version: 2020.06.15-nohash

Additionally, attach any relevant sections from the log files above. 

[lazka]

We delete packages that aren't used after 1.5 years.

[dalboris]

Oh wow, so you're saying no one had downloaded the package for one year before its deletion? This seems crazy, I wouldn't expect trying to install brotli or any package that depends on it on Windows 7 via vcpkg is so rare. But perhaps it is. Thanks for the answer anyway.

[lazka]

No :) it gets deleted 1.5 after it leaves the MSYS2 repository.

[dalboris]

Oh, I see, thanks for the clarification, this makes more sense. And just how of curiosity: how is the decision made to remove a package from the MSYS2 repository? I feel that keeping this one could be useful to quite a few people.

[lazka]

It's handled by this script: https://github.com/msys2/msys2-devtools/blob/main/msys2-repo-prune

I'm open to bump the limit a bit, but since we depend on sponsored servers/mirrors we can't let the it grow indefinitely.

[dalboris]

Thanks, yes, I understand that it's impossible (or at least, not reasonable without really good funding) to keep growing these indefinitely. I pretty much don't know nothing about how msys2 is funded/administered/maintained so I'll leave it up to your decision.

My data point is: without this package, vcpkg version 2020.11-1 is broken (at least for the package brotli and all its dependencies which I believe is a lot of packages), and any version of vcpkg version 2020.11-1 is broken too on Windows 7 due to Python 3.9+ not being supported on Windows 7.

I'll let you decide whether improving this state of affair is worth your server space availability.

[dalboris]

Just for info, I managed to get a working vcpkg on Windows 7 by using the latest vcpkg (so that it doesn't attempt to download a now-unavailable old version of pkg-config), but modifying it to force using Python 3.8.10 (instead of Python 9+ which doesn't work on Windows 7) by modifying some vcpkg files as explained here: https://github.com/microsoft/vcpkg/issues/20113#issuecomment-977463518

It's not ideal but at least there is a workaround.

[martingalvan-nordic]

We delete packages that aren't used after 1.5 years.

Got hit by a similar issue a couple of days ago. Where is this documented? Are these removals announced somewhere?

[lazka]

It isn't. We could document that users should update at least once every 1.5 years.

[martingalvan-nordic]

It isn't. We could document that users should update at least once every 1.5 years.

Yes, and it would be nice to announce these removals somewhere, as they can be quite breaking, esp. when relying on tools such as vcpkg. Thanks.

[lazka]

Yes, and it would be nice to announce these removals somewhere, as they can be quite breaking, esp. when relying on tools such as vcpkg. Thanks.

Do you have a suggestion on where we could document this? I'm not sure how vcpkg interacts with us and what place is good for that.

[martingalvan-nordic]

I don't have any suggestions, sorry.

[martingalvan-nordic]

The vcpkg maintainers knew about this issue and that's why they just updated the MSYS2 packages recently microsoft/vcpkg#22974

Yes, but I understand they had to update because they got hit by it. Knowing these things in advance would make things a bit smoother for everyone IMHO.

[martingalvan-nordic]

MSYS2 is a rolling-based platform so not being able to dowload old packages is very expected.

Which brings us back to my original point. Where is this "not being able to dowload old packages is very expected" officially documented/stated? I'm not saying it's not, I just want to know how this could be improved so that these issues will be more predictable.

[mati865]

Rolling policy is very briefly mentioned at https://www.msys2.org/docs/what-is-msys2/#msys2-vs-arch-linux I would gladly review PR that improves it. Perhaps Package Management section would be right?

[qdin]

We just got burned by this in a couple of different places.

We sometimes go back and build old release branches. For example, an old branch that works great needs a very minor change, and upgrading all dependent libraries would introduce a whole lot of noise - at least a thorough QA cycle and perhaps some code changes. In these cases we depend on the baseline / versioning feature of vcpkg and it has worked pretty great till this point.

I'm curious as to why it's necessary to delete stuff, esp that such a popular package manager depends on. Storage is so cheap these days. Not to be presumptuous about cost and affordability, but if you're already spending the money on mirroring, would be nice to keep at least one copy somewhere so vcpkg doesn't just fail and leave us hunting for a working baseline.

But perhaps events like this indicate that Vcpkg could use a bit more flexibility around its tools, or start providing its own storage for older versions of its dependent tools; i.e. keep baseline/versioining working even when old dependent tool packages disappear.

[lazka]

We delete packages that aren't used after 1.5 years.

This is now bumped to 1.75. We might increase it to 2 years in the future.

I've added a FAQ entry: https://github.com/msys2/msys2.github.io/commit/0b9134e2a67f050

[MehdiChinoune]

Not an MSYS2 issue.