EnvFile with PyCharm and pytest error: sys.path seems broken.

[jimklo]

So I create test file, test_stuff.py and define the following test:

import os


def test_read_env():
    endpoint_token = os.environ.get('MY_ENV_VAR')
    assert endpoint_token

I create a .env file with the following:

MY_ENV_VAR=1

Then I right click on the test file and select: "Run 'pytest in test_stuff...". Test runs, and fails as MY_ENV_VAR isn't defined.

I go an edit the test configuration, select the EnvFile tab, enable EnvFile plugin and add the .env file I created to the configuration. I click "Apply" and "OK".

I then re-run the test. I expect it should pass, instead... I get the following error:

Testing started at 12:44 PM ...
/usr/local/anaconda3/envs/access_35/bin/python "/Applications/PyCharm with Anaconda plugin .app/Contents/helpers/pycharm/_jb_pytest_runner.py" --target test_stuff.py::test_read_env
Launching pytest with arguments test_stuff.py::test_read_env in /Users/jklo/projects/SIBIS/source/hivalc-access2redcap/import_sources

============================= test session starts ==============================
platform darwin -- Python 3.5.5, pytest-3.8.1, py-1.8.0, pluggy-0.11.0 -- /usr/local/anaconda3/envs/access_35/bin/python
cachedir: .pytest_cache
rootdir: /Users/jklo/projects/SIBIS/source/hivalc-access2redcap/import_sources, inifile:
plugins: remotedata-0.3.1, openfiles-0.4.0, doctestplus-0.3.0, arraydiff-0.3
collecting ... collected 1 item

test_stuff.py::test_read_env 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/_pytest/main.py", line 236, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/_pytest/runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     item.ihook.pytest_runtest_logstart(nodeid=item.nodeid, location=item.location)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/anaconda3/envs/access_35/lib/python3.5/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Applications/PyCharm with Anaconda plugin .app/Contents/helpers/pycharm/teamcity/pytest_plugin.py", line 222, in pytest_runtest_logstart
INTERNALERROR>     self.ensure_test_start_reported(self.format_test_id(nodeid, location), test_name)
INTERNALERROR>   File "/Applications/PyCharm with Anaconda plugin .app/Contents/helpers/pycharm/teamcity/pytest_plugin.py", line 230, in ensure_test_start_reported
INTERNALERROR>     self.teamcity.testStarted(test_id, flowId=test_id, captureStandardOutput=capture_standard_output, metainfo=metainfo)
INTERNALERROR>   File "/Applications/PyCharm with Anaconda plugin .app/Contents/helpers/pycharm/_jb_runner_tools.py", line 179, in testStarted
INTERNALERROR>     commands = _TREE_MANAGER_HOLDER.manager.level_opened(self._test_to_list(testName), _write_start_message)
INTERNALERROR>   File "/Applications/PyCharm with Anaconda plugin .app/Contents/helpers/pycharm/_jb_runner_tools.py", line 50, in manager
INTERNALERROR>     self._fill_manager()
INTERNALERROR>   File "/Applications/PyCharm with Anaconda plugin .app/Contents/helpers/pycharm/_jb_runner_tools.py", line 58, in _fill_manager
INTERNALERROR>     from _jb_serial_tree_manager import SerialTreeManager
INTERNALERROR> ImportError: No module named '_jb_serial_tree_manager'

========================= no tests ran in 0.03 seconds =========================

Process finished with exit code 0

Empty suite

Empty suite

I'm not quite sure what's going on, but it seems like when EnvFile loads, it mangles the sys.path preventing the test runner from locating packages it needs to run.

Here's the troubleshooting info from my environment:

=== About ===
Build version: PyCharm 2019.2.1 Build #PY-192.6262.63 August 21, 2019
Java version: 11.0.3+12-b304.39x86_64
Operating System: Mac OS X (10.14.6, x86_64)
JVM version: OpenJDK 64-Bit Server VM JetBrains s.r.o

=== System ===
Number of CPU: 8
Used memory: 619Mb 
Free memory: 141Mb 
Total memory: 761Mb 
Maximum available memory: 1981Mb

=== Displays ===
Displays: 
Display 0: 1920x1200; scale: 1.00
Display 1: 6016x3384; scale: 2.00
Display 2: 3360x2100; scale: 2.00

=== Plugins ===
Custom plugins: [EnvFile (3.2.0), CSV Plugin (2.6.4), Datalore (0.1.2-192), .env files support (0.8)]
Disabled plugins:[IdeaVim (0.53), Rainbow CSV (1.1), Configuration Script (192.6262.63)]

[jimklo]

I believe I found the problem that is effecting me.

This could be construed as a "bug" or not... I'm not entirely sure what the right solution is here, as the current implementation is not portable across IDEs.

It turns out... I also had PYTHONPATH defined in my .env file.

According to the official Python documentation for PYTHONPATH, is used to augment the default search path for module files. It's not clear if this is append or replace.

This EnvFile plugin, which is essentially replicating functionality that exists in Webpack, VS Code, and other bundling solutions. With VS Code, which is an IDE as opposed to a bundler, it possible that EnvFiles should treat PYTHONPATH in a similar manner, where PYTHONPATH within .env specifies additional locations where the Python interpreter should look for modules.

[ashald]

Were you able to find a suitable workaround?

[dorli]

Changing the order of the .env files in the "EnvFile" configuration panel fix it for me. Just make your ".env" file first and "<Run Configuration Env Vars>" second.