samtools sourceforge download fails from open-uri: 'redirection forbidden'

[MatthewRalston]

Error building with ruby 2.1.10 due to open-uri. The mkrf_conf.rb and Rakefile both contain http sourceforge urls. These fail to download properly thanks to open-uri. I am surprised this wasn't tested. Running on Arch linux, ruby 2.1.10 with rvm. More information available on request.

>gem install ./bio-samtools-2.3.0.gem
Building native extensions.  This could take a while...
ERROR:  Error installing ./bio-samtools-2.3.0.gem:
        ERROR: Failed to build gem native extension.

    current directory: /home/matt/.rvm/gems/ruby-2.1.10@ngsci/gems/bio-samtools-2.3.0/ext
/home/matt/.rvm/rubies/ruby-2.1.10/bin/ruby mkrf_conf.rb

current directory: /home/matt/.rvm/gems/ruby-2.1.10@ngsci/gems/bio-samtools-2.3.0/ext
/home/matt/.rvm/rubies/ruby-2.1.10/bin/ruby -rubygems /home/matt/.rvm/gems/ruby-2.1.10@ngsci/gems/rake-12.2.1/exe/rake RUBYARCHDIR=/home/matt/.rvm/gems/ruby-2.1.10@ngsci/extensions/x86_64-linux/2.1.0/bio-samtools-2.3.0 RUBYLIBDIR=/home/matt/.rvm/gems/ruby-2.1.10@ngsci/extensions/x86_64-linux/2.1.0/bio-samtools-2.3.0
rake aborted!
redirection forbidden: http://sourceforge.net/projects/samtools/files/samtools/0.1.19/samtools-0.1.19.tar.bz2/download -> https://sourceforge.net/projects/samtools/files/samtools/0.1.19/samtools-0.1.19.tar.bz2/download
/home/matt/.rvm/gems/ruby-2.1.10@ngsci/gems/bio-samtools-2.3.0/ext/Rakefile:10:in `block in <top (required)>'
/home/matt/.rvm/gems/ruby-2.1.10@ngsci/gems/rake-12.2.1/exe/rake:27:in `<main>'
Tasks: TOP => default => download
(See full trace by running task with --trace)

rake failed, exit code 1

[homonecloco]

Hi, I'm a bit confused on which version are you trying to compile. I hope I don't have a mess of the configuration files, but you should be testing bio-samtools-2.5.1, as it is the latest version in the repository. The current version already works with samtools 1.x, and this seems to be trying to use samtools 0.1.19. Best, Ricardo.

[MatthewRalston]

URI-redirection issues happen across multiple versions of bio-samtools >2.3 that I tested. As shown in the text above, the version of bio-samtools i was installing at the time was 2.3.0, but I've encountered the issue in 2.3.2, 2.4, and 2.5.1. When you are testing, do you have open_uri_redirections installed or are you running on a fresh gemset every time?

v2.5.1 avoids the Config vs RbConfig syntax issue(#68), but it still doesn't avoid 2 types of URI redirection issues:

1. http://sourceforge.net/projects/samtools/files/samtools/0.1.19/samtools-0.1.19.tar.bz2/download => (301) https://sourceforge.net/projects/samtools/files/samtools/0.1.19/samtools-0.1.19.tar.bz2/download
2. (curl -Lv URL) https://sourceforge.net/projects/samtools/files/samtools/0.1.19/samtools-0.1.19.tar.bz2/download => (intermediates, 303/302) https://downloads.sourceforge.net/project/samtools/0.1.19/samtools-0.1.19.tar.bz2?r=&ts=1234567790&use_mirror=gigenet => (303/302 final mirror) https://gigenet.dl.sourceforge.net/project/samtools/samtools/0.1.19/samtools-0.1.19.tar.bz2

Second, v2.5.1 is still subject to htslib compilation issues (#66)

[homonecloco]

I usually check that the gem is passing the tests in travis-ci: https://travis-ci.org/helios/bioruby-samtools I'll try to have a look, but I can't promise when, as I have several commitments going to at the moment.

[MatthewRalston]

That's fine, I was able to eventually install the gem when I got the combination right. I'll try to provide some more reproducible gem version/ruby version combos. I am actually very surprised you haven't historically observed any of the uri-redirection issues from sourceforge.

In travis.ci, I'm not seeing any reference to the sourceforge download.... can anyone explain this? Also, why do you need the accessory commands instead of gem build bioruby-samtools.gemspec and gem install bioruby-samtools*.gem?

[homonecloco]

Thanks for the feedback. We did see the problems with the uri redirection, and it should have been fixed when we updated the code to work with samtools 1.x... I'll need to investigate further, I wonder if my fork is not properly in sync with the master here.