[bug] Race condition - parallel writing on deactive_conanbuildenv.sh with 2 parallel commands

[Bearwolves]

Environment details

• Operating System+version: Ubuntu 18.04
• Conan version: 1.56.0
• Python version: 3.6

Steps to reproduce

Let's assume, we have a recipe tool-x containing a system_requirements() which indeed check if a package is installed (in our case usbutils).

    name="tool-x"

    def system_requirements(self):
        apt = Apt(self, arch_names={"<conan_arch_setting>": "apt_arch_setting"})
        apt.install(["usbutils"], check=True)

In our CI pipeline, we run the conan test and a conan install of the created package in parallel as we want to test different things. In the first case, the package, in the second the usability of the tools by our consumers as we are using the deploy(). We were surprised that the pipeline fails sometimes. In fact, both calls sources the same conanbuildenv.sh! See logs. It leads that one may overwrite partially the other deactivate_conanbuildenv.sh which leads to single quote parsing issue in the shell interpreter, see logs.

I propose following patch to make this behavior explicitly failing (neither retrying nor waiting is in my opinion a good idea):

--- environment.py	2023-01-11 10:18:40.797067056 +0100
+++ environment.py	2023-01-11 10:19:32.869589199 +0100
@@ -378,6 +378,12 @@
         filepath, filename = os.path.split(file_location)
         deactivate_file = os.path.join(filepath, "deactivate_{}".format(filename))
         deactivate = textwrap.dedent("""\
+           if [ -f {deactivate_file}.lock ]; then
+               echo "ERROR: Trying to write to a file which is locked [{deactivate_file}]"
+               exit 1
+           fi
+           touch {deactivate_file}.lock
+           ls -al {deactivate_file}.lock
            echo "echo Restoring environment" > "{deactivate_file}"
            for v in {vars}
            do
@@ -390,6 +396,7 @@
                    echo unset $v >> "{deactivate_file}"
                fi
            done
+           rm {deactivate_file}.lock
            """.format(deactivate_file=deactivate_file, vars=" ".join(self._values.keys())))
         capture = textwrap.dedent("""\
               {deactivate}

What do you think?

Logs

Debugging setup

export CONAN_LOG_RUN_TO_OUTPUT =1
export CONAN_LOGGING_LEVEL=10
export CONAN_PRINT_RUN_COMMANDS=1
export CONAN_VERBOSE_TRACEBACK=1

Path setup

BUILD_DIR="${SCRIPT_DIR}/build/${PROFILE}"
TEST_BASE_DIR="${BUILD_DIR}-test"
TEST_DIR="${TEST_BASE_DIR}/conan"
DEPLOY_TEST_DIR="${TEST_BASE_DIR}/robot"

Parallel execution of following 2 commands and their relevant debugging output

conan test "${SCRIPT_DIR}/test/conan" "${COMPONENT_NAME}/${VERSION}@${REFERENCE}" \
--profile:build "linux-settings" \
--profile:host "${PROFILE}" \
--test-build-folder "${TEST_DIR}"

[2023-01-11T13:58:01.132Z] > . "/data/jenkins/workspace/tool-x_PR-13/build/linux-settings/conanbuild.sh" && dpkg-query -W -f='${Status}' usbutils | grep -q "ok installed"

conan install "${COMPONENT_NAME}/${VERSION}@${REFERENCE}" \
--generator VirtualRunEnv \
--install-folder "${DEPLOY_TEST_DIR}" \
--profile:host "${PROFILE}"

[2023-01-11T13:58:01.211Z] > . "/data/jenkins/workspace/tool-x_PR-13/build/linux-settings/conanbuild.sh" && dpkg-query -W -f='${Status}' usbutils | grep -q "ok installed"

Content of the deactivate_conanbuildenv.sh

echo Restoring environment
9xx/.conan/data/tools/0.19.0/user/stable/package/2d343e1b326e747f0a9e5f73ffa67a3ed62f2b2c/tools:'
export AAAAAAAAAAAA='/tmp/xxx9xx/.conan/data/tools-config/0.32.0/user/stable/package/7e808ba62e711d669863b5c5b26db60b51b690ca/src'
.....

Issue while running deactivate_conanbuildenv.sh

Restoring environment
/data/jenkins/workspace/COIN_tools-x_PR-13/build/linux-settings/deactivate_conanbuildenv.sh: line 11: unexpected EOF while looking for matching `''

[memsharded]

Hi @Bearwolves

Thanks for your report. @AbrilRBS is working on https://github.com/conan-io/conan/pull/12914, I think that would also solve race conditions.

[AbrilRBS]

Pinging @uilianries who I think will be happy to work in this with me in the future :)