Halide icon indicating copy to clipboard operation
Halide copied to clipboard

Package manager status

Open alexreinking opened this issue 4 years ago • 26 comments

This issue tracks the overall status of the various first-party and community-maintained packages out in the wild.

If you are interested in packaging Halide please reach out to us here on this issue.

Status: OK

Repository Version Maintainers
PyPI (Test) 🟢 19.0.0.dev38 @alexreinking
Debian package 🟢 18.0.0-8 @LebedevRI
Homebrew 🟢 18.0.0 @ankane
Arch AUR 🟢 18.0.0-1 @OmarEmaraDev
conda-forge 🟢 18.0.0 @alexreinking @isuruf
vcpkg 🟡 18.0.0 (pre-release) @alexreinking @yurybura

Status: Needs update

Repository Version Maintainers
Nix 🔴 16.0.0 @ck3d @twesterhout
Nuget ⚠️ unknown commit from 2018 @alexreinking

Status: TODO

Repository Issue / PR Status
Conan conan-io/conan-center-index#6890 🔴 no interest

alexreinking avatar Feb 28 '20 02:02 alexreinking

Uploading Python packages to PyPI for Linux distributions requires the code to be compilable on an old CentOS system (the "manylinux" distribution uses CentOS 6). Not sure if it is going to be a problem. In particular, I am not sure how challenging it is to compile the latest llvm version there. C++14 support might also be tricky (but not impossible).

BachiLi avatar Feb 28 '20 12:02 BachiLi

That's good to know. I wonder if there are Docker images with CentOS 6 and recent development tools. Otherwise, we could probably set up a buildbot with a custom built GCC for this. Painful, but done once, and probably worthwhile.

alexreinking avatar Feb 28 '20 18:02 alexreinking

That's good to know. I wonder if there are Docker images with CentOS 6 and recent development tools. Otherwise, we could probably set up a buildbot with a custom built GCC for this. Painful, but done once, and probably worthwhile.

Yes there are docker images: https://github.com/pypa/manylinux

BachiLi avatar Feb 28 '20 18:02 BachiLi

An old release is on vcpkg. See #4539 . We should work with them once #4644 lands to see what they would like out of our build.

alexreinking avatar May 05 '20 12:05 alexreinking

Hey @alexreinking, happy to help get this on Homebrew. I started a formula here. Two things that would help are:

  1. A make install task
  2. A new release. The latest release fails with error: no viable conversion from 'int' to 'llvm::MaybeAlign', but the current master branch builds successfully

ankane avatar Jul 21 '20 10:07 ankane

PR #5135 begins versioning Halide with semver at 10.0.0

alexreinking avatar Jul 28 '20 23:07 alexreinking

@ankane - a new release is imminent. Does Homebrew require make install specifically, or would cmake --install /builddir --prefix /installdir suffice?

alexreinking avatar Jul 28 '20 23:07 alexreinking

Awesome, cmake --install works just as well.

ankane avatar Jul 29 '20 00:07 ankane

@ankane - we just released Halide 10.0.0. Please let us know how to help you get it on Homebrew :)


I just reached out to the folks at vcpkg, too. See microsoft/vcpkg#13580

alexreinking avatar Sep 17 '20 06:09 alexreinking

Thanks @alexreinking! I've submitted it to Homebrew here: https://github.com/Homebrew/homebrew-core/pull/61246

ankane avatar Sep 17 '20 07:09 ankane

Awesome, @ankane! Keep us posted

alexreinking avatar Sep 17 '20 07:09 alexreinking

Time to look into Python packaging. I'll look into pip and conda. This means trying to build Halide in a manylinux docker image, and set up scripts for bots.

https://docs.python.org/3/c-api/stable.html

BachiLi avatar Sep 18 '20 20:09 BachiLi

Homebrew is live 🎉

brew install halide

ankane avatar Oct 01 '20 17:10 ankane

Amazing! Thanks, @ankane!

alexreinking avatar Oct 01 '20 17:10 alexreinking

Homebrew is live 🎉

brew install halide

:HAPPYDANCE:

steven-johnson avatar Oct 01 '20 17:10 steven-johnson

Vcpkg is live! 🎉

$ vcpkg install halide             # Linux, macOS
$ vcpkg install halide:x64-windows # Windows
$ vcpkg install halide[target-all] # include all backends, not just host

microsoft/vcpkg#13860

alexreinking avatar Oct 14 '20 07:10 alexreinking

what about https://pypi.org/project/halide/#history ? it seems that the previous release was in Jun 2013

tyoc213 avatar Jan 31 '21 15:01 tyoc213

We don't own that package. We should reach out to whoever created it and see if we can get it transferred.

alexreinking avatar Jan 31 '21 20:01 alexreinking

I think the package can meet

  • https://www.python.org/dev/peps/pep-0541/#how-to-request-a-name-transfer
  • https://www.python.org/dev/peps/pep-0541/#continue-maintenance
  • here is the list of what is "abandoned" https://www.python.org/dev/peps/pep-0541/#id23

And it is good that the old package and the new one is "the same" not a "different one". Also could not see how to contact the maintainer.

Is this one owned here? https://anaconda.org/conda-forge/halide and if needed a new name can be used instead of conda-forge.

tyoc213 avatar Jan 31 '21 22:01 tyoc213

Is this one owned here? https://anaconda.org/conda-forge/halide and if needed a new name can be used instead of conda-forge.

See #5609

alexreinking avatar Feb 01 '21 00:02 alexreinking

By the way ppl my sugiestion would be if you want if you can use https://github.com/fastai/pypi_template#how-to-use-the-templateI think it should be easy to add and it should work for pip and conda.

So that is my suguestion :).

tyoc213 avatar Feb 01 '21 06:02 tyoc213

@alexreinking I am creating a standard AUR package for Halide and I was wondering if you can take a look and provide some insights. I pushed an initial implementation to the AUR here. This is the PKGBUILD.

With regards to dependencies, are there any missing dependencies? Which dependencies are needed at compile time and which are needed at runtime?

The installed files are as follows. Does this structure look correct?

File list:
halide /usr/
halide /usr/bin/
halide /usr/bin/featurization_to_sample
halide /usr/bin/get_host_target
halide /usr/bin/retrain_cost_model
halide /usr/bin/weightsdir_to_weightsfile
halide /usr/include/
halide /usr/include/Halide.h
halide /usr/include/HalideBuffer.h
halide /usr/include/HalidePyTorchCudaHelpers.h
halide /usr/include/HalidePyTorchHelpers.h
halide /usr/include/HalideRuntime.h
halide /usr/include/HalideRuntimeCuda.h
halide /usr/include/HalideRuntimeD3D12Compute.h
halide /usr/include/HalideRuntimeHexagonDma.h
halide /usr/include/HalideRuntimeHexagonHost.h
halide /usr/include/HalideRuntimeMetal.h
halide /usr/include/HalideRuntimeOpenCL.h
halide /usr/include/HalideRuntimeOpenGLCompute.h
halide /usr/include/HalideRuntimeQurt.h
halide /usr/include/wasm-rt-impl.h
halide /usr/include/wasm-rt.h
halide /usr/lib/
halide /usr/lib/cmake/
halide /usr/lib/cmake/Halide/
halide /usr/lib/cmake/Halide/Halide-shared-deps.cmake
halide /usr/lib/cmake/Halide/Halide-shared-targets-release.cmake
halide /usr/lib/cmake/Halide/Halide-shared-targets.cmake
halide /usr/lib/cmake/Halide/HalideConfig.cmake
halide /usr/lib/cmake/Halide/HalideConfigVersion.cmake
halide /usr/lib/cmake/HalideHelpers/
halide /usr/lib/cmake/HalideHelpers/Halide-Interfaces-release.cmake
halide /usr/lib/cmake/HalideHelpers/Halide-Interfaces.cmake
halide /usr/lib/cmake/HalideHelpers/HalideGeneratorHelpers.cmake
halide /usr/lib/cmake/HalideHelpers/HalideHelpersConfig.cmake
halide /usr/lib/cmake/HalideHelpers/HalideHelpersConfigVersion.cmake
halide /usr/lib/cmake/HalideHelpers/HalideTargetHelpers.cmake
halide /usr/lib/libHalide.so
halide /usr/lib/libHalide.so.12
halide /usr/lib/libHalide.so.12.0.1
halide /usr/lib/libautoschedule_adams2019.so
halide /usr/lib/libautoschedule_li2018.so
halide /usr/lib/libautoschedule_mullapudi2016.so
halide /usr/lib/libwasm-rt-impl.a
halide /usr/share/
halide /usr/share/Halide/
halide /usr/share/Halide/README.md
halide /usr/share/Halide/README_cmake.md
halide /usr/share/Halide/README_rungen.md
halide /usr/share/Halide/README_webassembly.md
halide /usr/share/Halide/tools/
halide /usr/share/Halide/tools/GenGen.cpp
halide /usr/share/Halide/tools/RunGen.h
halide /usr/share/Halide/tools/RunGenMain.cpp
halide /usr/share/Halide/tools/autotune_loop.sh
halide /usr/share/Halide/tools/halide_benchmark.h
halide /usr/share/Halide/tools/halide_image.h
halide /usr/share/Halide/tools/halide_image_info.h
halide /usr/share/Halide/tools/halide_image_io.h
halide /usr/share/Halide/tools/halide_malloc_trace.h
halide /usr/share/Halide/tools/halide_trace_config.h
halide /usr/share/Halide/tools/mex_halide.m
halide /usr/share/licenses/
halide /usr/share/licenses/halide/
halide /usr/share/licenses/halide/LICENSE

OmarEmaraDev avatar Sep 04 '21 20:09 OmarEmaraDev

Hi @OmarEmaraDev! Thanks for putting together an AUR package!

With regards to dependencies, are there any missing dependencies? Which dependencies are needed at compile time and which are needed at runtime?

When you build against shared LLVM (as you are), then the libLLVM.so library is a runtime dependency. If the WebAssembly backend is enabled, then the LLD development package is a dependency to develop with Halide, but isn't a runtime dependency.

So your dependencies list looks good to me since you aren't splitting the package up into runtime and development components (in Debian conventions: libhalide and libhalide-dev). If you later go that route, just remember that lld is a dependency of libhalide-dev and not of libhalide.

The installed files are as follows. Does this structure look correct?

It seems fine at a glance, I would encourage you to try building the apps with your installed version of Halide. The PKGBUILD has one thing that looks odd to me, but I don't have any experience authoring packages for the AUR. This here:

  install -Dm644 "Halide/LICENSE.txt" "$pkgdir/usr/share/licenses/$pkgname/LICENSE"

I would have expected the value of $pkgdir to be something like /usr, so this would expand to /usr/usr/share/..., wouldn't it?

alexreinking avatar Sep 06 '21 00:09 alexreinking

I would encourage you to try building the apps with your installed version of Halide.

@alexreinking The apps were built successfully and their corresponding tests passed.

I disabled the cuda and linear algebra apps because I didn't have a development environment for those at the moment. Also, perhaps unrelated, abseil-cpp doesn't compile due to a missing include #include <limits> in abseil-cpp/absl/synchronization/internal/graphcycles.cc because of the use of std::numeric_limits<uint32_t>::max().

I would have expected the value of $pkgdir to be something like /usr, so this would expand to /usr/usr/share/..., wouldn't it?

The Arch build system uses this approach called the "fakeroot environment". Essentially, Arch creates some directory $pkgdir and tells you to pretend that this is the root of your file system, anything you install under this directory will be installed under the actual root / during package installation. So $pkgdir/usr/share/* will be installed in /usr/share/*.

OmarEmaraDev avatar Sep 06 '21 08:09 OmarEmaraDev

Also, perhaps unrelated, abseil-cpp doesn't compile due to [...]

I wasn't aware any of our apps depend on Abseil! I wouldn't worry about it, since it's just an app.

Essentially, Arch creates some directory $pkgdir and tells you to pretend that this is the root of your file system, anything you install under this directory will be installed under the actual root / during package installation

Got it! Thanks for the clarification.

alexreinking avatar Sep 07 '21 00:09 alexreinking

The table in first comment is rather dated :) https://tracker.debian.org/news/1503976/accepted-halide-1700-1-source-all-amd64-into-unstable/ Maybe will finally get RISCV64 build for -2.

LebedevRI avatar Feb 17 '24 00:02 LebedevRI