[libaec] Add new port

[bansan85]

Fixes #38073 #19125

Add libaec and replace szip by libaec in hdf5

Add libaec

• [X] The name of the port matches an existing name for this component on https://repology.org/ if possible, and/or is strongly associated with that component on search engines.
• [X] Optional dependencies are resolved in exactly one way. For example, if the component is built with CMake, all find_package calls are REQUIRED, are satisfied by vcpkg.json's declared dependencies, or disabled with CMAKE_DISABLE_FIND_PACKAGE_Xxx.
• [X] The versioning scheme in vcpkg.json matches what upstream says.
• [X] The license declaration in vcpkg.json matches what upstream says.
• [X] The installed as the "copyright" file matches what upstream says.
• [X] The source code of the component installed comes from an authoritative source.
• [X] The generated "usage text" is accurate. See adding-usage for context.
• [X] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• [X] Only one version is in the new port's versions file.
• [X] Only one version is added to each modified port's versions file.

Update hdf5

• [ ] Changes comply with the maintainer guide.
• [X] SHA512s are updated for each updated download.
• [ ] ~~The "supports" clause reflects platforms that may be fixed by this new version.~~
• [ ] ~~Any fixed CI baseline entries are removed from that file.~~
• [X] Any patches that are no longer applied are deleted from the port's directory.
• [X] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• [X] Only one version is added to each modified port's versions file.

[bansan85]

Need to see which and why other packages failed

[dg0yt]

vpckg is weak at handling alternatives, and libaec is an alternative to szip. Each file name can only be owned by one port. Switching some ports to libaec while szip is still used by others will not work well.

At least libaec is active and keeping up it is promise of providing a drop-in replacement. IDK if the vcpkg owners are willing to accept a full switch.

[bansan85]

From what I saw, hdf5 is the only one that use directly szip. One other package uses szip but only with hdf5. I made few PR and they all need some work. I will fix this PR within few days.

[bansan85]

Do you think I should remove szip package ?

[JonLiu1993]

Do you think I should remove szip package ?

Don't delete a port casually. This will cause problems for the ports that depend on it.

[JonLiu1993]

@bansan85, I want to test your features and usage locally, but I tried twice and failed to download libaec-v1.1.3.tar.gz correctly. Could you try the following locally and see if it can be downloaded normally?

PS F:\Feature-test\libaec> ./vcpkg install libaec:x64-windows
Computing installation plan...
The following packages will be built and installed:
    libaec:[email protected]
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
Restored 0 package(s) from C:\Users\v-zhli17\AppData\Local\vcpkg\archives in 184 us. Use --debug to see more details.
Installing 1/1 libaec:[email protected]...
Building libaec:[email protected]...
-- Downloading https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.tar.gz -> k202009-libaec-v1.1.3.tar.gz...
[DEBUG] To include the environment variables in debug output, pass --debug-env
[DEBUG] Trying to load bundleconfig from F:\Feature-test\libaec\vcpkg-bundle.json
[DEBUG] Failed to open: F:\Feature-test\libaec\vcpkg-bundle.json
[DEBUG] Bundle config: readonly=false, usegitregistry=false, embeddedsha=nullopt, deployment=Git, vsversion=nullopt
[DEBUG] Metrics enabled.
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] Feature flag 'dependencygraph' unset
Downloading https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.tar.gz
warning: Download failed -- retrying after 1000ms
warning: Download failed -- retrying after 2000ms
warning: Download failed -- retrying after 4000ms
error: Failed to download from mirror set
error: https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.tar.gz: WinHttpSendRequest failed with exit code 12002
error: https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.tar.gz: WinHttpSendRequest failed with exit code 12002
error: https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.tar.gz: WinHttpSendRequest failed with exit code 12002
error: https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.tar.gz: WinHttpSendRequest failed with exit code 12002
[DEBUG] D:\a\_work\1\s\src\vcpkg\base\downloads.cpp(1031):
[DEBUG] Time in subprocesses: 0us
[DEBUG] Time in parsing JSON: 2us
[DEBUG] Time in JSON reader: 0us
[DEBUG] Time in filesystem: 1131us
[DEBUG] Time in loading ports: 0us
[DEBUG] Exiting after 1.5 min (91181618us)

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:32 (message):

      Failed to download file with error: 1
      If you are using a proxy, please check your proxy setting. Possible causes are:

      1. You are actually using an HTTP proxy, but setting HTTPS_PROXY variable
         to `https://address:port`. This is not correct, because `https://` prefix
         claims the proxy is an HTTPS proxy, while your proxy (v2ray, shadowsocksr
         , etc..) is an HTTP proxy. Try setting `http://address:port` to both
         HTTP_PROXY and HTTPS_PROXY instead.

      2. If you are using Windows, vcpkg will automatically use your Windows IE Proxy Settings
         set by your proxy software. See https://github.com/microsoft/vcpkg-tool/pull/77
         The value set by your proxy might be wrong, or have same `https://` prefix issue.

      3. Your proxy's remote server is out of service.

      If you've tried directly download the link, and believe this is not a temporary
      download server failure, please submit an issue at https://github.com/Microsoft/vcpkg/issues
      to report this upstream download server failure.


Call Stack (most recent call first):
  scripts/cmake/vcpkg_download_distfile.cmake:270 (z_vcpkg_download_distfile_show_proxy_and_fail)
  scripts/cmake/vcpkg_from_gitlab.cmake:113 (vcpkg_download_distfile)
  ports/libaec/portfile.cmake:1 (vcpkg_from_gitlab)
  scripts/ports.cmake:175 (include)

[bansan85]

@JonLiu1993 https://dkrz.de and all sub-domain are down...

Lots of red in https://status.dkrz.de/

[bansan85]

@JonLiu1993 Website is back. But "We currently have serious issues with a central network switch causing failure of most of our virtual environment. This affects most of our services."

[dg0yt]

Don't delete a port casually. This will cause problems for the ports that depend on it.

This answer may be a little short. In the light of potential conflicts.

[JonLiu1993]

Tested libaec and hdf5 usage successfuly by libaec:x64-windows and hdf5:x64-windows

[JonLiu1993]

@bansan85, It seems that the above header files are installed in the wrong location. You need to take a look.

[dg0yt]

@bansan85, It seems that the above header files are installed in the wrong location. You need to take a look.

No. The point that already was made is that libaec is intented to be used as a drop-in replacement for szip. I cannot assess how much this is true. The full file list indicates that it might need additional adjustments. But it implies that vcpkg cannot build both in CI. Either we start to "skip" szip (as done with opencv2 or opencv2), or we make it an empty port which depends on libaec.

[bansan85]

I think the location of files are good. But it's true that there is a conflict with szip.

I replaced the deletion of unused library, patched CMakeLists.txt, rebase and remove usage file (it's not the same target for static and dynamic linkage).

[bansan85]

I just noticed caffe2:

portfile.cmake:

SET(VCPKG_POLICY_EMPTY_PACKAGE enabled)
message(WARNING "The 'caffe2' port is deprecated, using 'libtorch' instead")

vcpkg.json:

{
  "name": "caffe2",
  "version": "0.8.1",
  "port-version": 8,
  "description": "Caffe2 is a lightweight, modular, and scalable deep learning framework.",
  "homepage": "https://github.com/caffe2/caffe2",
  "license": "Apache-2.0",
  "dependencies": [
    {
      "name": "libtorch",
      "default-features": false
    }
  ]
}

Maybe I could increase port-version of szip and deprecate it ?

[dg0yt]

This is an example of "make it an empty port which depends on" the new port. This requires a (mostly) complete drop-in replacement.

The downside of this transition is that no new version of szip could be added to the vcpkg registry as long as it overlaps with libaec. OTOH there is no demand for szip in the vcpkg registry except for hdf5, and they prefer libaec IIRC. The benefit of choosing the preferred implementation is important enough IMO.

The alternative is to patch libaec and hdf5 to use non-conflicting names. And this would still come with extra efforts to avoid symbol clashes with msbuild autolinking. I hope this will not be asked for.

(You really take an adavanced hands-on course on vcpkg porting with this PR. :-)

[bansan85]

Sure I learned a lot :) but I didn't except to spend so much time to remove szip library.

I understand that szip and libaec are incompatible.

IMO, szip and libaec should not be installed at the same time. I like the idea of the deprecated szip in flavor of libaec. I think this should be done and wait if someone complains that he really want to use szip.

I can also add a new feature "libaec" to hdf5 and make it default instead of szip.

I'm pretty sure that (almost) nobody care about using libaec instead of szip (which is not patent-free).

[Neumann-A]

Relevant discussion: https://github.com/h5py/h5py/issues/706#issuecomment-227967665

[JonLiu1993]

Tested libaec and hdf5 usage successfuly by libaec:x64-windows and hdf5:x64-windows

[bansan85]

New week, better motivation :)

It's good to know that the patent has been removed.

I know you already answered but please confirm me which solution you want:

• Remove szip and use libaec,
• Fake szip and use libaec. Add a warning message if szip is used. If someone complains, implement one of the solutions below.
• Add support for both as feature (only one should be enabled at the same time). Both libraries can't be installed at the same time (no patch to rename library filename),
• Add support for both as feature (only one should be enabled at the same time). Both libraries can be installed at the same time (need to patch to rename library filename). Should I rename szip library or libaec library ?

[vicroms]

@bansan85

The team discussed this PR and we think that changing all uses of szip to libaec seems like the best path. Could you please make the necessary changes to replace all uses of szip and remove the port.

[JonLiu1993]

Tested libaec and hdf5 usage successfuly by libaec:x64-windows and hdf5:x64-windows

[bansan85]

This is not ready for merge.

Must-have: CMake config must support multi-config. (It is not an exported config!)

I don't get it.

For libaec, I have:

-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel

Is this for hdf5 that have 2 Configuring that should be merged into one ?

-- Configuring x64-windows-dbg
-- Configuring x64-windows-rel
-- Building x64-windows-dbg
-- Building x64-windows-rel

Nice-to-have: Trim the patch by not changing indentation.

So you would like I add if()/endif() and not indent inside it ? OK.

[dg0yt]

Must-have: CMake config must support multi-config. (It is not an exported config!)

It is not about how libaec is built. It is about the CMake config package installed by the port:

libaec:x64-windows:/share/libaec/libaec-config-version.cmake
libaec:x64-windows:/share/libaec/libaec-config.cmake

There are not the usual files with exported target properties for release and debug. AFAICS libaec-config.cmake is hand-written, as old find modules did. By using this config will not get multi-config variables and target properties. They may end up with mixing debug and release binaries, which means incompatible runtimes on Windows.

So either libaec-config.cmake needs to be patched (or replaced) for multi-config support (select_library_configurations, IMPORTED_LOCATION_<CONFIG>), or it is replaced with a proper export of the CMake targets. (The second option is the prefered solution also for upstreaming. But if you are not familiar with that, it might be a quite some effort.)

For reference, szip had a typical exported config package:

szip:x64-windows:/share/szip/szip-config-version.cmake
szip:x64-windows:/share/szip/szip-config.cmake
szip:x64-windows:/share/szip/szip-targets-debug.cmake
szip:x64-windows:/share/szip/szip-targets-release.cmake
szip:x64-windows:/share/szip/szip-targets.cmake

[bansan85]

@dg0yt I implemented the export of targets using CMake features and cleanup libaec-config.cmake.

If this is fine I will report in a second time the patch to upstream.