How to install the python library inside a virtualenv?

[munikarmanish]

After following the instructions in the INSTALL.md file, I can successfully import bcc from the system python interpreter (v3.6). But I cannot import bcc from the python within the virtualenv (v3.7). I want to write BPF program with some 3.7 features (time.time_ns) How can I install the python bcc library inside a virtualenv? I'm used to installing per-virtualenv python packages using pip, but this library doesn't seem to have that option. Thanks in advance. :)

[willfindlay]

Are you using pipenv? If so, you can install bcc normally and pass --site-packages as a flag to pipenv as a workaround. As far as I know, there is no good way to get bcc working properly with a virtual environment.

[photoszzt]

If you already created the virtual env, you can copy the bcc dir to your virtualenv's lib dir. I'm not sure whether symlink would work.

[gukoff]

You can also workaround this by importing bcc from the global location. Caveat is that different platforms will have a different "global location", and many platforms try to import from the places like "/usr/lib/python3.10/site-packages/bcc/__init__.py".

Example for Debian:

try:
    import bcc
except ImportError:
    spec = importlib.util.spec_from_file_location("bcc", "/usr/lib/python3/dist-packages/bcc/__init__.py")
    if spec:
        module = importlib.util.module_from_spec(spec)
        sys.modules['bcc'] = module
        spec.loader.exec_module(module)

[westurner]

Installing bcc with pip [from pypi] would be easier with a few updates to modernize the python packaging.

To support this;

pip install pybcc

pip install -e git+ssh://[email protected]/iovisor/bcc/__todo_subpath_/#egg=bcc

• [ ] pyproject.toml is the new way. setup.py is the old way and it's not backward compatible with putting all the config for all the tools in pyproject.toml.
  • https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
  • https://packaging.python.org/en/latest/guides/packaging-binary-extensions/#alternatives-to-handcoded-wrapper-modules
  • [ ] pick a PEP517-compliant build backends for CMake (given CMakeLists.txt)
    • https://github.com/scikit-build/scikit-build-core#example
    • https://github.com/tttapa/py-build-cmake#usage
    • https://github.com/cmake-wheel/cmeel#basic-pyprojecttoml-example
• [ ] decide/bikeshed on a unique name of the python package given the existing python module name bcc
  • pypi:bcc is already registered: https://pypi.org/project/bcc
  • pypi:pybcc is not registered: https://pypi.org/project/pybcc
  • a package named pybcc can install to site-packages/bcc/ so that import bcc works (e.g. pip install pdbpp preempts the pdb module with this method)
• cibuildwheel builds manylinux wheels and inlines all dependencies with auditwheel (or delocate) but IIRC it's also possible to use shared libraries?
• configure GitHub Actions with cibuildwheel
  • https://cibuildwheel.pypa.io/en/stable/ci-services/#github-actions
  • https://cibuildwheel.pypa.io/en/stable/deliver-to-pypi/#github-actions

pypa/cibuildwheel:

• https://github.com/pypa/cibuildwheel
• https://cibuildwheel.pypa.io/en/stable/#how-it-works
• https://cibuildwheel.pypa.io/en/stable/configuration/#configuration-file

pypa/manylinux:

• https://github.com/pypa/manylinux

pypa/auditwheel:

• https://github.com/pypa/auditwheel

github actions with cibuildwheel and attestations:

• https://github.com/marketplace/actions/build-and-inspect-a-python-package#inputs :

  attest-build-provenance-github: Whether to generate signed build provenance attestations for workflow artifacts using actions/attest-build-provenance. Requires attestations: write and id-token: write permissions. The only meaningful value is 'true' (note the quotes – GitHub Actions only allow string inputs) and everything else is treated as falsey. (optional, default: 'false').

  Important: GitHub's artifact attestations are different from PyPI's Sigstore attestations that you can generate while uploading using pypa/gh-action-pypi-publish

• https://github.com/hynek/build-and-inspect-python-package#usage

sources:

• https://github.com/iovisor/bcc/blob/master/src/python/setup.py.in
• https://github.com/iovisor/bcc/blob/master/src/python/CMakeLists.txt