C compiler does not work

[gkarthik]

Hello,

I'm trying to add a recipe that requires header files from htslib and so I'm downloading it and compiling it in build.sh. When I test it locally using circleci build I get the following error,

checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/opt/conda/conda-bld/ivar_1545294925462/work/htslib':
configure: error: C compiler cannot create executables

In my meta.yaml I have specified {{ compiler('c') }} in the requirements,

requirements:
  build:
    - {{ compiler('c') }}
    - {{ compiler('cxx') }}
    - autoconf
    - automake
    - libtool
    - pkg-config

This is the line with ./configure in build.sh where the error is thrown,

./configure --prefix=`pwd`
make install

I checked a couple of recipes including htslib and they all seem to have similar commands. I tested the recipe locally on Ubuntu 16.04 and MacOS HighSierra(just in case) and got the same results. I'm not sure what I'm missing here. Thank you!

Karthik

[bgruening]

Can you make sure that you don't have any left over from an old bioconda-utils?

[AlexanderDilthey]

@bgruening Confirmed - this is a fresh installation on a freshly set up virtual machine.

[bgruening]

@AlexanderDilthey thanks for testing I will try to find some time to reproduce this. For the time being please create direct PRs with your recipe and use this as testing. Of course, you can use simple conda build before if you like. Sorry for the inconvenience.

[gkarthik]

@tomkinsc suggested that a fix here is to add an ancillary conda_build_config.yaml to pin specific compilers. Spec is here. After pinning the compilers the build worked and I've submitted a pull request #12929 where all the tests on CircleCI have passed.

The {{compiler()}} jinja2 function is not fully functional is due to an upstream delay in conda-forge. Refer to this issue here.

[tomkinsc]

The addition of a conda_build_config.yaml does resolve the issue, but it's not entirely clear to me why the system compiler from devtoolset-2 in the docker image is not being picked up. Should the bioconda conda_build_config.yaml pin compilers in advance of the conda-forge migration to GCC7?

[AlexanderDilthey]

The addition of a conda_build_config.yaml resolved the issue for my cloned samtools recipe as well.

As a next step, I tried to package up a Perl CPAN module which also requires compilation.

I created a recipe with conda skeleton cpan Text::LevenshteinXS.

circleci build fails with a gcc error message:

gcc -c   -D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2   -DVERSION=\"0.03\" -DXS_VERSION=\"0.03\" -fPIC "-I/opt/conda/conda-bld/perl-text-levenshteinxs_1546887339994/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/5.26.2/x86_64-linux-thread-multi/CORE"   LevenshteinXS.c
/bin/sh: gcc: command not found

I've added a conda_build_config.yaml:

c_compiler:
  - gcc      # [linux]
  - clang    # [osx]
cxx_compiler:
  - gxx      # [linux]
  - clangxx  # [osx]

... but the error message still appears. I also tried to explicitly include compiler dependencies into meta.yaml:

{% set name = "perl-text-levenshteinxs" %}
{% set version = "0.03" %}
{% set sha256 = "e374ff7b237919ce5ea9245f356d1cb52cc87fd26b3a5a38b3f3e5ff82a01491" %}

package:
  name: {{ name }}
  version: {{ version }}

source:
  url: https://cpan.metacpan.org/authors/id/J/JG/JGOLDBERG/Text-LevenshteinXS-0.03.tar.gz
  sha256: {{ sha256 }}

# If this is a new build for the same version, increment the build
# number. If you do not include this key, it defaults to 0.
build:
  number: 0

requirements:
  build:
    - {{ compiler('c') }}
    - {{ compiler('cxx') }}
    
  host:
    - perl
    - perl-test
    - perl-exporter
    - perl-xsloader

  run:
    - perl
    - perl-test
    - perl-exporter
    - perl-xsloader

test:
  # Perl 'use' tests
  imports:
    - Text::LevenshteinXS

  # You can also put a file called run_test.pl (or run_test.py) in the recipe
  # that will be run at test time.

about:
  home: http://metacpan.org/pod/Text::LevenshteinXS
  license: unknown
  summary: 'An XS implementation of the Levenshtein edit distance'

# See
# http://docs.continuum.io/conda/build.html for
# more information about meta.yaml

Again, no success.

Any advice?

[gkarthik]

Could you please post build.sh? I'm trying to recreate it.

[AlexanderDilthey]

Sure, here we go - build.sh:

#!/bin/bash

# If it has Build.PL use that, otherwise use Makefile.PL
if [ -f Build.PL ]; then
    perl Build.PL
    perl ./Build
    perl ./Build test
    # Make sure this goes in site
    perl ./Build install --installdirs site
elif [ -f Makefile.PL ]; then
    # Make sure this goes in site
    perl Makefile.PL INSTALLDIRS=site
    make
    make test
    make install
else
    echo 'Unable to find Build.PL or Makefile.PL. You need to modify build.sh.'
    exit 1
fi

# Add more build steps here, if they are necessary.

# See
# http://docs.continuum.io/conda/build.html
# for a list of environment variables that are set during the build process.

[AlexanderDilthey]

Hi @gkarthik, any luck with this by any chance? Any way I can help?

[AlexanderDilthey]

Similar problems also occur for commits pushed to GitHub.

/bin/sh: /home/conda/feedstock_root/build_artifacts/perl_1537776081706/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc: No such file or directory

Pull request https://github.com/bioconda/bioconda-recipes/pull/13166

[AlexanderDilthey]

Addition of a conda_build_config.yaml also doesn't solve the problem.

[dpryan79]

It's quite annoying, but for perl packages you have to do something like this and this. I agree that this is incredibly annoying, but until someone comes up with a good way to make perl behave in a sane way hacks like this are needed.

[wshuai294]

I met a similar bug in https://github.com/bioconda/bioconda-recipes/issues/47103