debugging py_test targets with pytest main "hangs"

[rajukrishnamurthy]

Description of the bug:

pytest targets executed with pytest.main() seems to hang, but appears to be running stat() every file in the filesystem. The debugging session doesn't ever start because there are inevitably unreadable files with throws errors.

Which category does this issue belong to?

Intellij, PyCharm

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

a minimal repo can be found here

If you debug the unit_test target, I am able to set successfully debug. However, if you try to debug the pytest_test target, it "hangs" on with the folllowing:

import sys; print('Python %s on %s' % (sys.version, sys.platform))
/usr/bin/python3 "/Users/rkrishnamurthy/Library/Application Support/JetBrains/IntelliJIdea2024.2/plugins/python-ce/helpers/pydev/pydevd.py" --multiprocess --qt-support=auto --client 127.0.0.1 --port 51013 --file /private/var/tmp/_bazel_rkrishnamurthy/5303facb39f6d1f7ffd509576224e2f0/execroot/videoamp/bazel-out/darwin_arm64-fastbuild/bin/demo/python/idea_debug/py/py_test
Connected to pydev debugger (build 242.23339.11)
============================= test session starts ==============================
platform darwin -- Python 3.9.18, pytest-7.0.1, pluggy-1.0.0
rootdir: /

if you pause/restart the debugging session a few times, there are repeated calls to Path.stat() in lib/python3.9/pathlib.py

Which Intellij IDE are you using? Please provide the specific version.

IDEA ultimate 2024.2.3

What programming languages and tools are you using? Please provide specific versions.

python 3.9 toolchain

What Bazel plugin version are you using?

2024.09.10.0.1-api-version-242

Have you found anything relevant by searching the web?

There is an ongoing slack thread, however, the ability to debug py_test targets is new, so I suspect people haven't run into this yet.

Any other information, logs, or outputs that you want to share?

No response

[agluszak]

Hey @rajukrishnamurthy, could you verify if #6847 fixes the issue for you?

[rajukrishnamurthy]

@agluszak I pulled down master built and installed the plugin and am getting the same behavior with the repo I referenced above.

[tpasternak]

Oh, just found we've already received this report in #5146

[agluszak]

@agluszak I pulled down master built and installed the plugin and am getting the same behavior with the repo I referenced above.

It's not merged yet, so you need to build branch agluszak/fix-6824

[tpasternak]

@rajukrishnamurthy I have merged that, you can try master branch now

[rajukrishnamurthy]

Apologies, I didn't look at that closely enough. Okay, with the fix in place it is no longer scanning the entire filesystem, however it doesn't seem to actually run the test. I've noticed a couple of things:

1. when pytest.main() is called in my stripped down example, I can trace to pytest_cmdline_main return NO_TESTS_COLLECTED. Perhaps my main is incorrect?
2. We override the py_test macro where we set environment variables args that we use to pass to pytest.main(). It seems to bypass that entirely and use the py_test from rules_python. I've added an example with an override of py_test defined here. This is probably a separate problem, but I imagine many people do this.

In both cases, bazel test runs the tests correctly.

I'm happy to continue to be a tester and I will try to be more timely in my response.

Thanks!

[agluszak]

Regarding 1, I cannot reproduce the issues you're describing. I can normally debug all of your 3 examples. Please see the video.

https://github.com/user-attachments/assets/9bf7b9fb-6ecf-40a2-8f63-d948d08d284a

Note that I'm using a python run configuration, not a bazel run configuration.

Regarding 2, as far as I know (although I have very little experience with how Python is exactly implemented), Bazel rules are not at all involved in debugging and running tests if they are run as a python run configuration. They are probably run through IntelliJ's native test runner, not through Bazel. If you want to pass custom env variables, you can do so in the run configuration.

Or, if you want to run the test through Bazel, you can create a run config like this:

Here, I agree, something weird is going on. If you Run this run configuration, it will use the env specified in your custom rule. If you debug it, it will use the one set in the run configuration.

Could you please record a video of how you run the test?

[JeroenSchmidt]

Hi @agluszak @rajukrishnamurthy A few weeks ago I hit the "file not found" error when using py_test + debug with a pytest wrapper script (very similar to your example) and it lead me to some interesting findings. I was able to find a work around but the root cause fix should be implemented.

See my comment here: https://github.com/bazelbuild/intellij/issues/6821#issuecomment-2439697168 . You can ignore everything above the linked comment because my initial theory was wrong, I'll continue the thread here.

While debugging the problem, I noticed two things:

• The working path within the python pytest script was the system root / and not the sandbox root. This was resulting in "file not found errors". This appears to be a bug because when the test is run from bazel CLI the working path is the sandbox root not system root.
• Almost all the bazel related env values are missing which are normally present when test target is run via bazel CLI
  • It looks like my local machine ENV are visible from the running pytest wrapper script when run through the plugin

I was able to fix the "file not found" error by extracting the sandbox root from the absolute path of the python pytest wrapper script located in the sandbox (other methods of getting the sandbox root didn't work because the env variables wasn't showing up).

import glob
import contextlib


@contextlib.contextmanager
def new_cd(x):
    d = os.getcwd()

    # This could raise an exception, but it's probably
    # best to let it propagate and let the caller
    # deal with it, since they requested x
    os.chdir(x)

    try:
        yield

    finally:
        os.chdir(d)

if __name__ == "__main__":
    import os
    import sys

    import pytest

    # NOTE: RUNFILES_DIR isn't present when this script is called via the intelij plugin. I had to comment it all out. 
    # runfiles = os.environ.get("RUNFILES_DIR")
    # logic with runfiles

    if os.environ.get("PYCHARM_HOSTED"): 
        # sys.argv[0] this has the location of pytest_test.py within the sandbox.
        path_to_script = sys.argv[0].replace("custom_main/pytest_test.py", "")

        with new_cd(path_to_script): # change the working directory to the sandbox root 
            sys.exit(pytest.main(sys.argv[1:]))
    else:
        # Normal case when running a bazel test from the cli "bazel test ..."
        sys.exit(pytest.main(sys.argv[1:]))

Config:

• plugin=2024.10.08.0.1-api-version-242
• bazel=7.3.1
• macos=14.6.1
• IntelliJ IDEA 2024.2.4
• rules_python==0.36.0

[tpasternak]

cc @sellophane

[github-actions[bot]]

Thank you for contributing to the IntelliJ repository! This issue has been marked as stale since it has not had any activity in the last 6 months. It will be closed in the next 14 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.