Building and distribute bwa together with bwapy

[1pakch]

I think it would be preferable to have an option to build and distribute bwa itself together with bwapy. There are two advantages:

• A possibility to get exactly the same version of the aligner itself in the $PATH when bwapy gets installed (+ for reproducible environments managed via conda, virualenv or similar).
• A possibility to create self-contained tests for the package

[cjw85]

I agree in principle that this is a nice option I have.

Currently this package does not exist on pypi (and I doubt we will have the time to put in the work to put it there - setuptools is a bit clunky when it comes to including arbitrary executables into the bin directory of a distribution). The repo contains the bwa source as a submodule, which can be used to build a bwa executable if desired.

If someone wants to make the build more elaborate we would accept pull requests, as I say I don't think this is something we will add.

[1pakch]

Ok. The use case I am thinking about are virtualenvs and possibly user's site-packages.

Would be good to know what are issues with putting executables in $PREFIX/bin from setuptools if I work on that. Could you please elaborate?

[cjw85]

IIRC there is no option to setuptools.setup to place items into dist/bin (listing binaries under scripts is a no go). One workaround is to do what is done here: https://github.com/nanoporetech/pomoxis/blob/master/setup.py, that is place pre-compiled executable files into the distribution as data, and then cp them after the fact into the the same place as where setuptools knows to place scripts.

[Permafacture]

MANIFEST.ini is the way to include non python files in your package at whatever location you want. But one would probably want to include the source and have setup.py compile it, no? Are there any issues with including the bwa source in this repo, rather than (or in addition to) as a submodule?

[cjw85]

MANIFEST.in controls files that get placed into the sdist of a python package, it will not (I believe) say anything about installation of built packages (what setup.py install achieves).

The bwa source could certainly be included in the sdist and distributed on pypi; though as I indicated a while back this isn't something we currently have an interest in doing. Should anyone wish to pick up maintenance and development of this project I would encourage them to create a fork.

[Permafacture]

the include_package_data kwarg in setup.py reads MANIFEST.ini and installs those files too.

[cjw85]

@Permafacture as per the OPs original specification, the issue at hand is installing the files to a location on the $PATH, do you know of a cleaner way to do this than the method used in the linked setup.py above using package data?

[Permafacture]

Oh, well, the goal was to package bwa with bwapy. OP mentioned putting a binary in $PATH as their idea of the solution. I don't think that's necessary though. When you compile your cffi objects through setup.py, you specify the paths to the resources you're using. I've only done this with source code (including the source of the project I'm linking to and then having setup.py/cffi build that project and my wrappers) but I'm pretty sure you can point to shared object files if that's preferred.

SO answer to linking to non-installed .so files: https://stackoverflow.com/questions/47508459/loading-so-library-with-cffi