rules_py

rules_py copied to clipboard

What happened?

Calling a py_binary in a subprocess fails if called with extra env vars. I prepared a demonstration in this branch: https://github.com/oqton/rules_py/tree/feature/subprocess_env_bug.

Here is the output I get:

bazelisk test //py/tests/binary_calls_binary/second
INFO: Analyzed target //py/tests/binary_calls_binary/second:second (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
FAIL: //py/tests/binary_calls_binary/second:second (see /home/mvukov/.cache/bazel/_bazel_mvukov/b62d0fb48c016b2423b665eb9db4b100/execroot/aspect_rules_py/bazel-out/k8-fastbuild/testlogs/py/tests/binary_calls_binary/second/second/test.log)
INFO: From Testing //py/tests/binary_calls_binary/second:second:
==================== Test output for //py/tests/binary_calls_binary/second:second:
ERROR: cannot find bazel_tools/tools/bash/runfiles/runfiles.bash
Traceback (most recent call last):
  File "/home/mvukov/.cache/bazel/_bazel_mvukov/b62d0fb48c016b2423b665eb9db4b100/sandbox/linux-sandbox/27/execroot/aspect_rules_py/bazel-out/k8-fastbuild/bin/py/tests/binary_calls_binary/second/second.runfiles/aspect_rules_py/py/tests/binary_calls_binary/second/second_main.py", line 3, in <module>
    subprocess.check_call('py/tests/binary_calls_binary/first/first', env={'FOO': 'bar'})
  File "/home/mvukov/.cache/bazel/_bazel_mvukov/b62d0fb48c016b2423b665eb9db4b100/execroot/aspect_rules_py/external/python_toolchain_x86_64-unknown-linux-gnu/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'py/tests/binary_calls_binary/first/first' returned non-zero exit status 1.
================================================================================
Target //py/tests/binary_calls_binary/second:second up-to-date:
  bazel-bin/py/tests/binary_calls_binary/second/second
INFO: Elapsed time: 1.692s, Critical Path: 1.27s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed, 1 test FAILED, 2 total actions
//py/tests/binary_calls_binary/second:second                             FAILED in 0.6s
  /home/mvukov/.cache/bazel/_bazel_mvukov/b62d0fb48c016b2423b665eb9db4b100/execroot/aspect_rules_py/bazel-out/k8-fastbuild/testlogs/py/tests/binary_calls_binary/second/second/test.log

INFO: Build completed, 1 test FAILED, 2 total actions

Without env flags in the subprocess the test succeeds.

Version

Development (host) and target OS/architectures:

Output of bazel --version: 5.3.0

Version of the Aspect rules, or other relevant rules from your WORKSPACE or MODULE.bazel file:

rules_py: based on c89d9481d52d277e12cf0c88aa8c1c72a76500c7.

Language(s) and/or frameworks involved:

Python

How to reproduce

# Call this when the aforementioned branch is checked out.
bazelisk test //py/tests/binary_calls_binary/second

Any other information?

No response

Fund our work

• [ ] Sponsor our open source work by donating a bug bounty

It turns out, the error is on my end, per documentation of subprocess.run. If env is specified it overrides the caller process env. So, something like to following fixes the test:

import os
import subprocess

env = os.environ
env.update({'FOO': 'bar'})
subprocess.check_call('py/tests/binary_calls_binary/first/first', env=env)

The old, non-working version, was:

import subprocess

subprocess.check_call('py/tests/binary_calls_binary/first/first', env={'FOO': 'bar'})

Nevertheless, its strange that I get ERROR: cannot find bazel_tools/tools/bash/runfiles/runfiles.bash error as the pwd essentially stays the same when the second app calls the first app in my example.

I'm not clear if there's work for us to do here, or this can be closed?

Well, at first the behavior of rules_py in this case was unexpected (works with rules_python :) ). Then I learned that my code was incomplete. Once I fixed it, then everything is fine. Let's close this.