Enable more consistent activation when using virtualenv's python binary directly

[follower]

As mentioned in #1080 the current documentation, to me, implies that using an activation script and then executing python functions the same as running /<virtualenv>/bin/python directly--which is not the case because the latter does not modify any environment variables:

If you directly run a script or the python interpreter from the virtualenv's bin/ directory (e.g. path/to/ENV/bin/pip or /path/to/ENV/bin/python-script.py) there's no need for activation.

(And, my experience with virtualenv and Ansible inventory scripts described in #1080 gave an example of where this difference in behavior caused hard-to-debug issues.)

It was stated in https://github.com/pypa/virtualenv/issues/906#issuecomment-244394963 that this behavior is intentional:

the Python executable doesn't set the PATH or VIRTUAL_ENV environment variables. That's by design, and is not a bug.

The underlying reason given for this intentional behavior is:

I don;'t want python in a virtualenv to work differently than the system python.

This intrigued me because I'd assumed that virtualenv already worked differently to system Python in order implement its functionality, so now I wondered how the functionality of changing the sys.path was implemented.

AFAICT this functionality is implemented within https://github.com/pypa/virtualenv/blob/master/virtualenv_embedded/site.py. And from this I learned about the potential for a sitecustomize module (e.g. site-packages/sitecustomize.py) as described in https://docs.python.org/2/library/site.html.

And, while a process can't modify its parent process's environment variables, it can modify its own environment variables and these will be inherited by any subprocesses.

With this discovery I created /<virtualenv path>/lib/python2.7/site-packages/sitecustomize.py based on activate_this.py:

import os
import sys 

old_os_path = os.environ.get('PATH', '')
os.environ['PATH'] = os.path.dirname(os.path.abspath(sys.executable)) + os.pathsep + old_os_path

(Clearly this is just a quick proof-of-concept, to be robust the code would need to avoid duplication of paths and also modify the other environment variables.)

And with this file in place I was able to run the original command (that uses the Amazon EC2 inventory script as per http://docs.ansible.com/ansible/latest/intro_dynamic_inventory.html#example-aws-ec2-external-inventory-script) successfully:

   ansible -i ec2.py --list-hosts all

Note:

• In my case Ansible had been installed via a recent version of Homebrew.

• These are relevant details about the installation:

  $ which ansible
  /usr/local/bin/ansible
  
  $ head -1 `which ansible`
  #!/usr/local/Cellar/ansible/2.3.2.0_1/libexec/bin/python
  

• It's possible to test this functionality without a Amazon EC2 account--you'll just get a boto.exception.NoAuthHandlerFound: No handler was ready to authenticate error instead of the expected list of hosts.

This functionality is definitely useful for me (in terms of creating a more "isolated 'virtual' python environment" for Python-based applications as Homebrew uses it) and given the comments on #906 it seems to have potential for others as well.

Have I missed any unintended side-effects from using this approach? Is using the sitecustomize.py approach consistent with the desire to remain consistent with the system Python functionality?

Even if deemed not consistent enough to include in virtualenv it seems documenting this approach would be helpful for those who wish to implement it themselves.

Thanks for considering this proposal.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.

[muelli]

This looks like a beautiful solution to the problem.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.

[gaborbernat]

@follower can you raise the issue upstream too? (aka in cpython bug tracker; would be interesting to see what the venv maintainers thing about this).

[follower]

Unfortunately I'm unlikely to get to doing that any time soon but would welcome anyone else who wishes to do so. :)

[gaborbernat]

Seems no longer an issue as no one reported it in 3 years.