horton icon indicating copy to clipboard operation
horton copied to clipboard

Remove context.py

Open tovrstra opened this issue 8 years ago • 3 comments

Our context.py is currently our approach to locate installed data files (test data files, basis sets, ...), which we install nicely in the FHS-designated $PREFIX/share directory. (See http://www.pathname.com/fhs/) This has a few problems though:

  1. The solution is non-modular. When we start to split of packages, every single one will have a context.py (if the package has data files).
  2. It duplicates some functionality already present in Python (and setuptools) to access installed data files.
  3. It does not work on .egg installations.

There are a few alternatives:

  • Bad solution, similar to context.py: use __file__ to locate data files.
  • Very basic: https://docs.python.org/3/library/pkgutil.html#pkgutil.get_data This just loads some file and returns a binary string with the content of the file.
  • Very elaborate but robust, and adds a dependency to setuptools: http://setuptools.readthedocs.io/en/latest/setuptools.html#accessing-data-files-at-runtime

tovrstra avatar Jul 04 '17 05:07 tovrstra

I've gained some more experience with this. The setuptools option works very well. These are the required steps:

  • Put all files that HORTON or its tests need inside the package directory, e.g. in a data subdirectory.
  • Include all these files in MANIFEST.in
  • Replace distutils to setuptools in setup.py
  • Replace Cython.Distutils by Cython.Build in setup.py
  • Set include_package_data=True when calling the setup function. Remove all other references to data files in setup.py.
  • Remove my_install_data from setup.py
  • Remove horton.context
  • Whenever you need an installed data file in HORTON, use the pkg_resources module, instead of horton.context. The module pkg_resources comes with setuptools. There are two ways of using it:
    • Efficient: http://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access This is to be preferred inside the HORTON library itself, e.g. in horton.periodic.
    • Easier: http://setuptools.readthedocs.io/en/latest/pkg_resources.html#resource-extraction This could be useful when a real file is needed somewhere, e.g. when testing if IOData can recognize the type of file from its name. In general this is OK for tests.

tovrstra avatar Aug 03 '17 07:08 tovrstra

P.S. Depending on setuptools is not a big deal. It is omnipresent these days, e.g installed when you have pip.

tovrstra avatar Aug 03 '17 07:08 tovrstra

A potential drawback is that nosetests does not find the tests when the package is installed as an Egg. The egg can be disabled in setup.py by adding an argument zip_safe=False to the setup() call.

tovrstra avatar Aug 03 '17 10:08 tovrstra