atest task loses python interpreter when running with virtualenv under Windows

[emanlove]

For issues

Steps to reproduce the issue

Under Windows

mkdir atest-python-interpreter
cd atest-python-interpreter
virtualenv -p C:\Python39\python.exe venv-py39
venv-py39\Scripts\activate
git clone [email protected]:robotframework/SeleniumLibrary.git rf-sl
cd rf-sl
pip install -r requirements-dev.txt
inv atest

The last command results in an error

(venv-py39) D:\devsandbox\atest-python-interpreter\rf-sl>inv atest
Traceback (most recent call last):
  File "D:\devsandbox\atest-python-interpreter\rf-sl\utest\run.py", line 9, in <module>
    from pytest import main as py_main
ModuleNotFoundError: No module named 'pytest'
Not running acceptance test, because unit tests failed.

Error messages and additional information

The last command results in an error

(venv-py39) D:\devsandbox\atest-python-interpreter\rf-sl>inv atest
Traceback (most recent call last):
  File "D:\devsandbox\atest-python-interpreter\rf-sl\utest\run.py", line 9, in <module>
    from pytest import main as py_main
ModuleNotFoundError: No module named 'pytest'
Not running acceptance test, because unit tests failed.

I am running under Windows 10 using Python 3.9.9 within a virtual env

Expected behavior and actual behavior

I was expecting the runner to use the virtual env I have executed with. But instead it appeas to run python from a process not realizing it is a virtualenv that it was started under.

Environment

Browser: Name and version (Usually available from the about dialogue.) Browser driver: Name and version Operating System: Windows 10 Libraries

• Robot Framework: 5.0.1
• Selenium: 4.5.0
• SeleniumLibrary: 6.1.0.dev1
• Interpreter: Python 3.9.9 under a virtualenv

(venv-py39) C:\projects\SeleniumLibrary\atest-python-interpreter\rf-sl>pip list
Package                      Version
---------------------------- -----------
[ ... snip ...]
pytest                       7.1.3
pytest-approvaltests         0.2.4
pytest-mockito               0.0.4
[ ... snip ...]

[aaltat]

Most likely replacing python with sys.executable could solve the problem. Also it most likely would be compatible with other OS.

[emanlove]

Was just submitting a pull request with that change :)

[aaltat]

Heh, I was faster. But I do not have Windows available for testing.

[emanlove]

Interesting .. I have a similar change but in a different spot .. typing commit notes now ..

[aaltat]

Most likely this should be fixed too: https://github.com/robotframework/SeleniumLibrary/blame/master/tasks.py#L211

[emanlove]

I was changing it at https://github.com/robotframework/SeleniumLibrary/blob/master/atest/run.py#L280 and then added a check to make sure it was not None nor an empty strip as per docs.

[emanlove]

So tried this within tasks.py

@task
def which(ctx):
    """Outputs which python."""
    command = "python -c \"import os, sys; print(os.path.dirname(sys.executable))\""
    ctx.run(command)

and it seems like tasks is getting the right interpreter. Where the issue seems to arise was when subprocess was calling python not knowing the code calling subprocess was under the virtual env.

[emanlove]

As I recheck the other subprocess calls within atest/run/py most are covered by my change. The one exception is the http_server method which uses "python". Adding a change for this one.

[rasjani]

So tried this within tasks.py

@task
def which(ctx):
    """Outputs which python."""
    command = "python -c \"import os, sys; print(os.path.dirname(sys.executable))\""
    ctx.run(command)

and it seems like tasks is getting the right interpreter. Where the issue seems to arise was when subprocess was calling python not knowing the code calling subprocess was under the virtual env.

that almost feels like that environment variables from the caller is not inherited to the sub process ?

[emanlove]

From what I am reading @rasjani is right in that just setting the python interpreter as sys.executable is not the same as activating a python virtualenv. The later also sets up some environment variables. So my solution is incomplete as written ..

[emanlove]

Was looking through the selenium v4.5.0 api trying to understand and resolve another problem but saw on this class a parameter called env which looks like a similar situation of needing to pass along env variables. I think in addition to the python interpreter I can set the env variables so that this should complete / match the operation of activating a python virtualenv.