ANTs icon indicating copy to clipboard operation
ANTs copied to clipboard

Build issues with conda-forge package

Open ghisvail opened this issue 4 years ago • 27 comments

I am in the process of submitting a conda-forge package for ANTs as part of my involvement with the Clinica project.

I got to a point where I have got a good recipe and tried a build by the conda-forge builders. It fails with a bunch of template argument deduction/substitution failed failures, such as:

2021-06-02T13:29:41.4891265Z In file included from $PREFIX/include/ITK-5.2/itkTransform.h:25,
2021-06-02T13:29:41.4891882Z                  from $PREFIX/include/ITK-5.2/itkMatrixOffsetTransformBase.h:24,
2021-06-02T13:29:41.4892524Z                  from $PREFIX/include/ITK-5.2/itkRigid3DTransform.h:22,
2021-06-02T13:29:41.4893149Z                  from $PREFIX/include/ITK-5.2/itkEuler3DTransform.h:22,
2021-06-02T13:29:41.4893776Z                  from $PREFIX/include/ITK-5.2/itkCenteredEuler3DTransform.h:22,
2021-06-02T13:29:41.4894267Z                  from ../ImageRegistration/itkANTSImageTransformation.h:24,
2021-06-02T13:29:41.4894751Z                  from ../ImageRegistration/itkPICSLAdvancedNormalizationToolKit.h:20,
2021-06-02T13:29:41.4895200Z                  from ../Examples/ANTS.cxx:18:
2021-06-02T13:29:41.4896356Z $PREFIX/include/ITK-5.2/itkVariableLengthVector.h:1301:1: note: candidate: 'template<class TExpr1, class TExpr2> typename itk::mpl::EnableIf<itk::Details::op::CanBeMultiplied<TExpr1, TExpr2>, itk::VariableLengthVectorExpression<TExpr1, TExpr2, itk::Details::op::Mult> >::Type itk::operator*(const TExpr1&, const TExpr2&)'
2021-06-02T13:29:41.4897302Z  1301 | operator*(TExpr1 const & lhs, TExpr2 const & rhs)
2021-06-02T13:29:41.4897656Z       | ^~~~~~~~
2021-06-02T13:29:41.4898281Z $PREFIX/include/ITK-5.2/itkVariableLengthVector.h:1301:1: note:   template argument deduction/substitution failed:
2021-06-02T13:29:41.4898799Z In file included from ../Utilities/ReadWriteData.h:10,
2021-06-02T13:29:41.4899236Z                  from ../ImageRegistration/ANTS_affine_registration2.h:32,
2021-06-02T13:29:41.4899698Z                  from ../ImageRegistration/itkANTSImageTransformation.cxx:22,
2021-06-02T13:29:41.4900223Z                  from ../ImageRegistration/itkANTSImageTransformation.h:190,
2021-06-02T13:29:41.4900701Z                  from ../ImageRegistration/itkPICSLAdvancedNormalizationToolKit.h:20,
2021-06-02T13:29:41.4901147Z                  from ../Examples/ANTS.cxx:18:
2021-06-02T13:29:41.4901802Z $PREFIX/include/ITK-5.2/itkImageFileWriter.h:254:88: note:   candidate expects 2 arguments, 1 provided
2021-06-02T13:29:41.4902391Z   254 |   using ImageType = typename std::remove_const<typename std::remove_reference<decltype(*image)>::type>::type;
2021-06-02T13:29:41.4902972Z       |   

The complete build log is available here.

ghisvail avatar Jun 02 '21 14:06 ghisvail

I'm not familiar with building in conda, but it looks like an ITK version compatibility. Would it be possible to do a Superbuild?

Also, if you can check out source code with git (even over https, no need to log in), it will copy accurate version information into the binaries.

cookpa avatar Jun 02 '21 19:06 cookpa

but it looks like an ITK version compatibility.

Perhaps something to do with how ITK 5 is built in the Superbuild vs conda-forge?

Would it be possible to do a Superbuild?

The conda-forge package should reuse existing dependencies available in the archive.

if you can check out source code with git

The conda-forge guidelines favor release tarballs over git checkouts.

Right now, I am using the release tarball from the latest tag (v2.3.5).

ghisvail avatar Jun 02 '21 19:06 ghisvail

You'll need to override the version info during build if you use a tarball: https://github.com/ANTsX/ANTs/issues/1183

gdevenyi avatar Jun 02 '21 20:06 gdevenyi

The version numbering is troublesome. This is definitely something we need to improve.

Building with system ITK is difficult because we tend to keep up to date with ITK development releases. You can get the version ANTs is currently using here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake

Sometimes you can find a nearby minor release of ITK that works.

cookpa avatar Jun 03 '21 15:06 cookpa

Le jeu. 3 juin 2021 à 17:49, Philip Cook @.***> a écrit :

The version numbering is troublesome. This is definitely something we need to improve.

Indeed. Maybe just maintain a Version.cmake without your versioning logic and let your CI process update the numbers on each release?

Letting downstream distributions handling the versioning themselves is fragile and a waste of resources for all stakeholders, imo.

Building with system ITK is difficult because we tend to keep up to date with ITK development releases. You can get the version ANTs is currently using here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake

Sometimes you can find a nearby minor release of ITK that works.

I see you are using ITK v5.2, the conda version is v5.2 and the conda version is maintained by the ITK community.

I may be wrong, but I believe the issue likely lies somewhere else.

ghisvail avatar Jun 03 '21 17:06 ghisvail

What prevents ANTs for being built with a standard release of ITK v5.2?

ghisvail avatar Jun 30 '21 21:06 ghisvail

As well as the version, there's specific options to the ITK build that ANTs needs.

In the SuperBuild, these are set up here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake

in particular see the options here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake#L114-L138

cookpa avatar Jul 01 '21 01:07 cookpa

I have contacted the ITK conda package maintainers, who helped identify a few issues. One may be that the conda compilers now default to C++17 and ITK and other third-party dependencies are built with this standards version.

Did you test any build of ANTs with C++17 in your CI infrastructure (with ITK built with C++17 too)? The template deduction errors here are in core ITK modules, which suggest a C++ standards mismatch.

ghisvail avatar Jul 14 '21 21:07 ghisvail

Thanks for this information. Our CI is pretty minimal. As far as I know, they just do the SuperBuild with the default options, including -std=c++11.

cookpa avatar Jul 15 '21 14:07 cookpa

Possibly relevant PR https://github.com/ANTsX/ANTs/pull/1209

cookpa avatar Jul 16 '21 13:07 cookpa

Further code updates in https://github.com/ANTsX/ANTs/pull/1218

cookpa avatar Aug 06 '21 12:08 cookpa

It builds fine now on both osx and linux targets. Do I still need ANTSPATH set if everything is installed under PATH already?

ghisvail avatar Aug 06 '21 16:08 ghisvail

The executables don't need it, but it's needed for many of the scripts.

cookpa avatar Aug 06 '21 16:08 cookpa

Ok, the good news are that ANTs now builds fine with C++17 on the conda-forge infrastructure with the conda toolchain.

Last details to be sorted concern the availability of a release tarball which contains the most recent fixes and how to properly pass version information to the build system. Could you please tag a new version of ANTs, and provide instructions as to how to set the appropriate version for the conda-forge executables?

Cheers, and thank you to all of you who helped make this happen :+1:

ghisvail avatar Aug 08 '21 15:08 ghisvail

All credit to @hjmjohnson

We're overdue for a new release tag, so I will work on that.

cookpa avatar Aug 08 '21 15:08 cookpa

@cookpa do you plan to make a new release soon then?

ghisvail avatar Aug 16 '21 12:08 ghisvail

I think #1216 should be a blocker for another release

gdevenyi avatar Aug 16 '21 12:08 gdevenyi

Yes, though I think I can fix that relatively quickly. I'll try to work on it this week.

cookpa avatar Aug 16 '21 18:08 cookpa

OK #1216 is done. I think we still need to address the version problem. I'll open a new issue

cookpa avatar Aug 26 '21 14:08 cookpa

I was wondering whether this https://anaconda.org/aramislab/ants is related to these efforts. What is the status of this issue?

/cc @arokem, who tipped me about the conda package.

oesteban avatar Jul 01 '22 12:07 oesteban

that's interesting - I never heard of that before.

stnava avatar Jul 01 '22 12:07 stnava

Yes, we used to maintain our own Conda channel for neuroimaging tools we needed for Clinica. We have contributed official conda-forge recipes for some of them, but the efforts for ANTs died out since the release promised last year has yet to happen.

I think the corresponding PR on conda-forge was also closed automatically due to inactivity.

ghisvail avatar Jul 01 '22 22:07 ghisvail

Does conda need ANTs to build against a release version of ITK? I ask because ITK 5.3.0 will be out soon, so if it makes sense to build a release against that, we could do so.

cookpa avatar Jul 05 '22 16:07 cookpa

Does conda need ANTs to build against a release version of ITK? I ask because ITK 5.3.0 will be out soon, so if it makes sense to build a release against that, we could do so.

It needs to build against whatever version of ITK is available in their distribution. Today it is 5.2, one day it will be 5.3.

ghisvail avatar Jul 05 '22 16:07 ghisvail

If you're agreeable @stnava @ntustison I shall bump to 2.4.0, since it's been a while and there are quite a few changes.

cookpa avatar Jul 05 '22 17:07 cookpa

yup

stnava avatar Jul 05 '22 20:07 stnava

https://github.com/ANTsX/ANTs/releases/tag/v2.4.0

cookpa avatar Jul 06 '22 15:07 cookpa