helpers icon indicating copy to clipboard operation
helpers copied to clipboard
verified publisher icon causify-ai

Break in slow tests - dockerized executables

Open gpsaggese opened this issue 8 months ago • 9 comments

GP Saggese

Friday at 5:43 PM @Sonya Nikiforova same glitch-y test linters/test/test_amp_dev_scripts.py::Test_linter_py1::test_linter_md1

That's a completely different error than before. Now it's failing because of some problem using the dockerized prettier. I'm guessing, it's because in this PR the default behavior of dev_scripts_helpers/documentation/lint_notes.py was changed from using prettier installed on the host to using the dockerized executable. To be more precise, the option to use prettier installed on the host was removed completely. BTW the PR was merged even though the slow tests were failing in it. Is there a way to completely block merging a PR if it breaks fast/slow tests? This is not the first time the build breaks because a PR was merged without accounting for the fact that it doesn't pass the fast/slow test checks.

FYI @dremdem @sonniki

gpsaggese avatar Apr 13 '25 17:04 gpsaggese

BTW the PR was merged even though the slow tests were failing in it. Is there a way to completely block merging a PR if it breaks fast/slow tests?

I think there are some settings already in place: Image

It's possible they are already configured, but there might be a bypass with your username: Image

dremdem avatar Apr 14 '25 10:04 dremdem

That's a completely different error than before. Now it's failing because of some problem using the dockerized prettier. I'm guessing, it's because in https://github.com/causify-ai/helpers/pull/544 the default behavior of dev_scripts_helpers/documentation/lint_notes.py was changed from using prettier installed on the host to using the dockerized executable. To be more precise, the option to use prettier installed on the host was removed completely.

This PR https://github.com/causify-ai/helpers/pull/558 should fix the slow test

  1. The issue is related to the mounting path of the source dir in the dockerized executables. It looks like we can't use the sibling container approach here. The children container approach should work. We can use the hserver.use_docker_sibling_containers() to check if sibling container is supported.
  2. I have enabled this dockerized executables tests on our CI as it doesn't take too long to run. LMK if we should disable it like the others.

FYI @gpsaggese @sonniki

user_1042@f8811eb590fd:/app$ python3 ./linters/amp_lint_md.py README.md
13:54:58 - INFO  hdbg.py init_logger:1018                               > cmd='./linters/amp_lint_md.py README.md'
13:55:01 - ERROR hsystem.py _system:252
################################################################################
cmd='($(find -wholename '*dev_scripts_helpers/documentation/lint_notes.py') -i README.md --in_place) 2>&1' failed with rc='1'
################################################################################
Output of the failing command is:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
13:55:01 - INFO  hdbg.py init_logger:1018                               > cmd='./dev_scripts_helpers/documentation/lint_notes.py -i README.md --in_place'
13:55:01 - ERROR hsystem.py _system:252
################################################################################
cmd='(docker run --rm --user $(id -u):$(id -g) -e AM_CONTAINER_VERSION -e CSFY_AWS_ACCESS_KEY_ID -e CSFY_AWS_DEFAULT_REGION -e CSFY_AWS_PROFILE -e CSFY_AWS_S3_BUCKET -e CSFY_AWS_SECRET_ACCESS_KEY -e CSFY_AWS_SESSION_TOKEN -e CSFY_CI -e CSFY_ECR_BASE_PATH -e CSFY_ENABLE_DIND -e CSFY_FORCE_TEST_FAIL -e CSFY_GIT_ROOT_PATH -e CSFY_HELPERS_ROOT_PATH -e CSFY_HOST_GIT_ROOT_PATH -e CSFY_HOST_NAME -e CSFY_HOST_OS_NAME -e CSFY_HOST_USER_NAME -e CSFY_HOST_VERSION -e CSFY_REPO_CONFIG_CHECK -e CSFY_REPO_CONFIG_PATH -e CSFY_TELEGRAM_TOKEN -e CSFY_USE_HELPERS_AS_NESTED_MODULE -e OPENAI_API_KEY  --entrypoint '' --workdir /app --mount type=bind,source=/data/heanhs/src/helpers2,target=/app tmp.prettier.x86_64.0ea6c25d bash -c "/usr/local/bin/prettier --parser markdown --prose-wrap always --tab-width 2 --print-width 80 --write /app/tmp.lint_notes.prettier.txt") 2>&1' failed with rc='125'
################################################################################
Output of the failing command is:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/heanhs/src/helpers2

Run 'docker run --help' for more information

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 409, in <module>
    _main(args_)
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 393, in _main
    txt = _process(
          ^^^^^^^^^
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 311, in _process
    txt = prettier_on_str(txt)
          ^^^^^^^^^^^^^^^^^^^^
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 143, in prettier_on_str
    prettier(tmp_file_name, tmp_file_name, *args, **kwargs)
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 119, in prettier
    hdocker.run_dockerized_prettier(
  File "/app/helpers/hdocker.py", line 670, in run_dockerized_prettier
    hsystem.system(docker_cmd)
  File "/app/helpers/hsystem.py", line 309, in system
    rc, _ = _system(
            ^^^^^^^^
  File "/app/helpers/hsystem.py", line 263, in _system
    raise RuntimeError(msg)
RuntimeError:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
_system failed: cmd='(docker run --rm --user $(id -u):$(id -g) -e AM_CONTAINER_VERSION -e CSFY_AWS_ACCESS_KEY_ID -e CSFY_AWS_DEFAULT_REGION -e CSFY_AWS_PROFILE -e CSFY_AWS_S3_BUCKET -e CSFY_AWS_SECRET_ACCESS_KEY -e CSFY_AWS_SESSION_TOKEN -e CSFY_CI -e CSFY_ECR_BASE_PATH -e CSFY_ENABLE_DIND -e CSFY_FORCE_TEST_FAIL -e CSFY_GIT_ROOT_PATH -e CSFY_HELPERS_ROOT_PATH -e CSFY_HOST_GIT_ROOT_PATH -e CSFY_HOST_NAME -e CSFY_HOST_OS_NAME -e CSFY_HOST_USER_NAME -e CSFY_HOST_VERSION -e CSFY_REPO_CONFIG_CHECK -e CSFY_REPO_CONFIG_PATH -e CSFY_TELEGRAM_TOKEN -e CSFY_USE_HELPERS_AS_NESTED_MODULE -e OPENAI_API_KEY  --entrypoint '' --workdir /app --mount type=bind,source=/data/heanhs/src/helpers2,target=/app tmp.prettier.x86_64.0ea6c25d bash -c "/usr/local/bin/prettier --parser markdown --prose-wrap always --tab-width 2 --print-width 80 --write /app/tmp.lint_notes.prettier.txt") 2>&1'
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
truncated output=
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/heanhs/src/helpers2

Run 'docker run --help' for more information


<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
  File "/app/./linters/amp_lint_md.py", line 114, in <module>
    _main(_parse())
  File "/app/./linters/amp_lint_md.py", line 110, in _main
    action.run(args.files)
  File "/app/linters/action.py", line 90, in run
    output = self.execute(file_name, pedantic=0)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/action.py", line 73, in execute
    output = self._execute(file_name, pedantic)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./linters/amp_lint_md.py", line 71, in _execute
    _, output = liutils.tee(cmd_as_str, self._executable, abort_on_error=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/utils.py", line 109, in tee
    rc, output = hsystem.system_to_string(cmd, abort_on_error=abort_on_error)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/helpers/hsystem.py", line 350, in system_to_string
    rc, output = _system(
                 ^^^^^^^^
  File "/app/helpers/hsystem.py", line 263, in _system
    raise RuntimeError(msg)
RuntimeError:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
_system failed: cmd='($(find -wholename '*dev_scripts_helpers/documentation/lint_notes.py') -i README.md --in_place) 2>&1'
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
truncated output=
13:55:01 - INFO  hdbg.py init_logger:1018                               > cmd='./dev_scripts_helpers/documentation/lint_notes.py -i README.md --in_place'
13:55:01 - ERROR hsystem.py _system:252
################################################################################
cmd='(docker run --rm --user $(id -u):$(id -g) -e AM_CONTAINER_VERSION -e CSFY_AWS_ACCESS_KEY_ID -e CSFY_AWS_DEFAULT_REGION -e CSFY_AWS_PROFILE -e CSFY_AWS_S3_BUCKET -e CSFY_AWS_SECRET_ACCESS_KEY -e CSFY_AWS_SESSION_TOKEN -e CSFY_CI -e CSFY_ECR_BASE_PATH -e CSFY_ENABLE_DIND -e CSFY_FORCE_TEST_FAIL -e CSFY_GIT_ROOT_PATH -e CSFY_HELPERS_ROOT_PATH -e CSFY_HOST_GIT_ROOT_PATH -e CSFY_HOST_NAME -e CSFY_HOST_OS_NAME -e CSFY_HOST_USER_NAME -e CSFY_HOST_VERSION -e CSFY_REPO_CONFIG_CHECK -e CSFY_REPO_CONFIG_PATH -e CSFY_TELEGRAM_TOKEN -e CSFY_USE_HELPERS_AS_NESTED_MODULE -e OPENAI_API_KEY  --entrypoint '' --workdir /app --mount type=bind,source=/data/heanhs/src/helpers2,target=/app tmp.prettier.x86_64.0ea6c25d bash -c "/usr/local/bin/prettier --parser markdown --prose-wrap always --tab-width 2 --print-width 80 --write /app/tmp.lint_notes.prettier.txt") 2>&1' failed with rc='125'
################################################################################
Output of the failing command is:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/heanhs/src/helpers2

Run 'docker run --help' for more information

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 409, in <module>
    _main(args_)
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 393, in _main
    txt = _process(
          ^^^^^^^^^
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 311, in _process
    txt = prettier_on_str(txt)
          ^^^^^^^^^^^^^^^^^^^^
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 143, in prettier_on_str
    prettier(tmp_file_name, tmp_file_name, *args, **kwargs)
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 119, in prettier
    hdocker.run_dockerized_prettier(
  File "/app/helpers/hdocker.py", line 670, in run_dockerized_prettier
    hsystem.system(docker_cmd)
  File "/app/helpers/hsystem.py", line 309, in system
    rc, _ = _system(
            ^^^^^^^^

heanhsok avatar Apr 14 '25 19:04 heanhsok

The fix from @heanhsok is good, thank you for jumping on to fix the build. As a side note, I don't understand why we had to disable the option in dev_scripts_helpers/documentation/lint_notes.py to run prettier on a host system directly, given that prettier is already installed in the helpers image anyway

sonniki avatar Apr 14 '25 19:04 sonniki

By the way, along with the tests, this change has also broken the Linter GH Action - but it's only visible when we are committing Markdown files. One of the Linter steps that is applied to Markdown files uses dev_scripts_helpers/documentation/lint_notes.py, and now it fails on GH Actions:

################################################################################
cmd='($(find -wholename '*dev_scripts_helpers/documentation/lint_notes.py') -i docs/work_organization/all.contributor_feedback.how_to_guide.md --in_place) 2>&1' failed with rc='1'
################################################################################
Output of the failing command is:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
09:23:17 - INFO  hdbg.py init_logger:1018                               > cmd='./dev_scripts_helpers/documentation/lint_notes.py -i docs/work_organization/all.contributor_feedback.how_to_guide.md --in_place'
09:23:17 - WARN  hdocker.py build_container_image:387                   Building Docker container...
  ... 2025/04/15 13:23:17 in: []string{}
  ... 2025/04/15 13:23:17 Parsed entitlements: []
  ... #0 building with "default" instance using docker driver
  ... 
  ... #1 [internal] load build definition from Dockerfile
  ... #1 transferring dockerfile: 252B done
  ... #1 DONE 0.0s
  ... 
  ... #2 [internal] load metadata for docker.io/library/node:18
  ... #2 DONE 0.1s
  ... 
  ... #3 [internal] load .dockerignore
  ... #3 transferring context: 2B done
  ... #3 DONE 0.0s
  ... 
  ... #4 [1/3] FROM docker.io/library/node:18@sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e
  ... #4 resolve docker.io/library/node:18@sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e done
  ... #4 sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e 6.41kB / 6.41kB done
  ... #4 sha256:aa6c239d30ee04dede270729f9502389b1a9546687ce656872536340ee0a9e03 2.49kB / 2.49kB done
  ... #4 sha256:de20d623379fc7c7ccf845a22c3153b920d57446ba7c8e64ba25d21a60b48ad6 6.39kB / 6.39kB done
  ... #4 sha256:23b7d26ef1d294256da0d70ce374277b9aab5ca683015073316005cb63d33849 48.49MB / 48.49MB 0.6s done
  ... #4 sha256:07d1b5af933d2dfc3d0dd509d6e20534825e4a537f7b006a6cb5b8e5a1f20905 24.01MB / 24.01MB 0.4s done
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 40.89MB / 64.40MB 0.6s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 17.83MB / 211.33MB 0.6s
  ... #4 sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f 0B / 3.32kB 0.6s
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 64.40MB / 64.40MB 0.7s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 32.51MB / 211.33MB 0.7s
  ... #4 sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f 3.32kB / 3.32kB 0.7s done
  ... #4 extracting sha256:23b7d26ef1d294256da0d70ce374277b9aab5ca683015073316005cb63d33849 0.1s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 0B / 45.68MB 0.7s
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 64.40MB / 64.40MB 0.8s done
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 48.23MB / 211.33MB 0.8s
  ... #4 sha256:04deb1529fda049f44f9be8d16ca833a53961813ce07eda3cefd50cd3fd74880 0B / 1.25MB 0.8s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 67.11MB / 211.33MB 0.9s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 6.29MB / 45.68MB 0.9s
  ... #4 sha256:04deb1529fda049f44f9be8d16ca833a53961813ce07eda3cefd50cd3fd74880 1.25MB / 1.25MB 0.8s done
  ... #4 sha256:3b3ca5178f3ece5f7e96b38b9c4b9c3a101a9d6777cf0f5c320f869512c80024 0B / 448B 0.9s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 80.74MB / 211.33MB 1.0s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 14.68MB / 45.68MB 1.0s
  ... #4 sha256:3b3ca5178f3ece5f7e96b38b9c4b9c3a101a9d6777cf0f5c320f869512c80024 448B / 448B 0.9s done
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 100.66MB / 211.33MB 1.1s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 23.07MB / 45.68MB 1.1s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 128.97MB / 211.33MB 1.3s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 36.70MB / 45.68MB 1.3s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 45.68MB / 45.68MB 1.4s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 158.33MB / 211.33MB 1.5s
  ... #4 sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 45.68MB / 45.68MB 1.4s done
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 171.97MB / 211.33MB 1.6s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 184.55MB / 211.33MB 1.7s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 196.08MB / 211.33MB 1.8s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 209.72MB / 211.33MB 1.9s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 211.33MB / 211.33MB 2.1s done
  ... #4 extracting sha256:23b7d26ef1d294256da0d70ce374277b9aab5ca683015073316005cb63d33849 1.9s done
  ... #4 extracting sha256:07d1b5af933d2dfc3d0dd509d6e20534825e4a537f7b006a6cb5b8e5a1f20905
  ... #4 extracting sha256:07d1b5af933d2dfc3d0dd509d6e20534825e4a537f7b006a6cb5b8e5a1f20905 0.4s done
  ... #4 extracting sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7
  ... #4 extracting sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 2.0s done
  ... #4 extracting sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621
  ... #4 extracting sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 5.0s done
  ... #4 extracting sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f done
  ... #4 extracting sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a
  ... #4 extracting sha256:058db40e534297246fb14bc4e107d2f6ddc140494a26793e1e90696cd6b2507a 1.7s done
  ... #4 extracting sha256:04deb1529fda049f44f9be8d16ca833a53961813ce07eda3cefd50cd3fd74880
  ... #4 extracting sha256:04deb1529fda049f44f9be8d16ca833a53961813ce07eda3cefd50cd3fd74880 4.0s done
  ... #4 extracting sha256:3b3ca5178f3ece5f7e96b38b9c4b9c3a101a9d6777cf0f5c320f869512c80024 done
  ... #4 DONE 29.4s
  ... 
  ... #5 [2/3] RUN npm install -g prettier
  ... #5 6.139 
  ... #5 6.139 added 1 package in 610ms
  ... #5 6.139 
  ... #5 6.139 1 package is looking for funding
  ... #5 6.139   run `npm fund` for details
  ... #5 6.141 npm notice
  ... #5 6.141 npm notice New major version of npm available! 10.8.2 -> 11.3.0
  ... #5 6.141 npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.3.0
  ... #5 6.141 npm notice To update run: npm install -g [email protected]
  ... #5 6.141 npm notice
  ... #5 DONE 6.2s
  ... 
  ... #6 [3/3] WORKDIR /app
  ... #6 DONE 3.7s
  ... 
  ... #7 exporting to image
  ... #7 exporting layers
  ... #7 exporting layers 1.0s done
  ... #7 writing image sha256:541c72e899ba241113f39a9ab2dee62d40152730140052334347e8f072111cfd done
  ... #7 naming to docker.io/library/tmp.prettier.x86_64.0ea6c25d done
  ... #7 DONE 1.0s
09:23:58 - INFO  hdocker.py build_container_image:407                   Building Docker container... done
09:23:58 - ERROR hsystem.py _system:252             
################################################################################
cmd='(docker run --rm --user $(id -u):$(id -g) -e AM_CONTAINER_VERSION -e CSFY_AWS_ACCESS_KEY_ID -e CSFY_AWS_DEFAULT_REGION -e CSFY_AWS_PROFILE -e CSFY_AWS_S3_BUCKET -e CSFY_AWS_SECRET_ACCESS_KEY -e CSFY_AWS_SESSION_TOKEN -e CSFY_CI -e CSFY_ECR_BASE_PATH -e CSFY_ENABLE_DIND -e CSFY_FORCE_TEST_FAIL -e CSFY_GIT_ROOT_PATH -e CSFY_HELPERS_ROOT_PATH -e CSFY_HOST_GIT_ROOT_PATH -e CSFY_HOST_NAME -e CSFY_HOST_OS_NAME -e CSFY_HOST_USER_NAME -e CSFY_HOST_VERSION -e CSFY_REPO_CONFIG_CHECK -e CSFY_REPO_CONFIG_PATH -e CSFY_TELEGRAM_TOKEN -e CSFY_USE_HELPERS_AS_NESTED_MODULE -e OPENAI_API_KEY  --entrypoint '' --workdir /app --mount type=bind,source=/home/runner/work/helpers/helpers,target=/app tmp.prettier.x86_64.0ea6c25d bash -c "/usr/local/bin/prettier --parser markdown --prose-wrap always --tab-width 2 --print-width 80 --write /app/tmp.lint_notes.prettier.txt") 2>&1' failed with rc='125'
################################################################################
Output of the failing command is:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /home/runner/work/helpers/helpers
Run 'docker run --help' for more information

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 409, in <module>
    _main(args_)
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 393, in _main
    txt = _process(
          ^^^^^^^^^
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 311, in _process
    txt = prettier_on_str(txt)
          ^^^^^^^^^^^^^^^^^^^^
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 143, in prettier_on_str
    prettier(tmp_file_name, tmp_file_name, *args, **kwargs)
  File "/app/./dev_scripts_helpers/documentation/lint_notes.py", line 119, in prettier
    hdocker.run_dockerized_prettier(
  File "/app/helpers/hdocker.py", line 670, in run_dockerized_prettier
    hsystem.system(docker_cmd)
  File "/app/helpers/hsystem.py", line 309, in system
    rc, _ = _system(
            ^^^^^^^^
  File "/app/helpers/hsystem.py", line 263, in _system
    raise RuntimeError(msg)
RuntimeError: 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
_system failed: cmd='(docker run --rm --user $(id -u):$(id -g) -e AM_CONTAINER_VERSION -e CSFY_AWS_ACCESS_KEY_ID -e CSFY_AWS_DEFAULT_REGION -e CSFY_AWS_PROFILE -e CSFY_AWS_S3_BUCKET -e CSFY_AWS_SECRET_ACCESS_KEY -e CSFY_AWS_SESSION_TOKEN -e CSFY_CI -e CSFY_ECR_BASE_PATH -e CSFY_ENABLE_DIND -e CSFY_FORCE_TEST_FAIL -e CSFY_GIT_ROOT_PATH -e CSFY_HELPERS_ROOT_PATH -e CSFY_HOST_GIT_ROOT_PATH -e CSFY_HOST_NAME -e CSFY_HOST_OS_NAME -e CSFY_HOST_USER_NAME -e CSFY_HOST_VERSION -e CSFY_REPO_CONFIG_CHECK -e CSFY_REPO_CONFIG_PATH -e CSFY_TELEGRAM_TOKEN -e CSFY_USE_HELPERS_AS_NESTED_MODULE -e OPENAI_API_KEY  --entrypoint '' --workdir /app --mount type=bind,source=/home/runner/work/helpers/helpers,target=/app tmp.prettier.x86_64.0ea6c25d bash -c "/usr/local/bin/prettier --parser markdown --prose-wrap always --tab-width 2 --print-width 80 --write /app/tmp.lint_notes.prettier.txt") 2>&1'
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
truncated output=
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /home/runner/work/helpers/helpers

Run 'docker run --help' for more information


<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/venv/lib/python3.12/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 598, in __call__
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/app/./linters/base.py", line 354, in _lint
    cur_action_lints = action_class.execute(file_path, pedantic)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/action.py", line 73, in execute
    output = self._execute(file_name, pedantic)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/amp_lint_md.py", line 71, in _execute
    _, output = liutils.tee(cmd_as_str, self._executable, abort_on_error=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/utils.py", line 109, in tee
    rc, output = hsystem.system_to_string(cmd, abort_on_error=abort_on_error)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/helpers/hsystem.py", line 350, in system_to_string
    rc, output = _system(
                 ^^^^^^^^
  File "/app/helpers/hsystem.py", line 263, in _system
    raise RuntimeError(msg)
RuntimeError: 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
_system failed: cmd='($(find -wholename '*dev_scripts_helpers/documentation/lint_notes.py') -i docs/work_organization/all.contributor_feedback.how_to_guide.md --in_place) 2>&1'
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
truncated output=
09:23:17 - INFO  hdbg.py init_logger:1018                               > cmd='./dev_scripts_helpers/documentation/lint_notes.py -i docs/work_organization/all.contributor_feedback.how_to_guide.md --in_place'
09:23:17 - WARN  hdocker.py build_container_image:387                   Building Docker container...
  ... 2025/04/15 13:23:17 in: []string{}
  ... 2025/04/15 13:23:17 Parsed entitlements: []
  ... #0 building with "default" instance using docker driver
  ... 
  ... #1 [internal] load build definition from Dockerfile
  ... #1 transferring dockerfile: 252B done
  ... #1 DONE 0.0s
  ... 
  ... #2 [internal] load metadata for docker.io/library/node:18
  ... #2 DONE 0.1s
  ... 
  ... #3 [internal] load .dockerignore
  ... #3 transferring context: 2B done
  ... #3 DONE 0.0s
  ... 
  ... #4 [1/3] FROM docker.io/library/node:18@sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e
  ... #4 resolve docker.io/library/node:18@sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e done
  ... #4 sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e 6.41kB / 6.41kB done
  ... #4 sha256:aa6c239d30ee04dede270729f9502389b1a9546687ce656872536340ee0a9e03 2.49kB / 2.49kB done
  ... #4 sha256:de20d623379fc7c7ccf845a22c3153b920d57446ba7c8e64ba25d21a60b48ad6 6.39kB / 6.39kB done
  ... #4 sha256:23b7d26ef1d294256da0d70ce374277b9aab5ca683015073316005cb63d33849 48.49MB / 48.49MB 0.6s done
  ... #4 sha256:07d1b5af933d2dfc3d0dd509d6e20534825e4a537f7b006a6cb5b8e5a1f20905 24.01MB / 24.01MB 0.4s done
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 40.89MB / 64.40MB 0.6s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 17.83MB / 211.33MB 0.6s
  ... #4 sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f 0B / 3.32kB 0.6s
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 64.40MB / 64.40MB 0.7s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 32.51MB / 211.33MB 0.7s
  ... #4 sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f 3.32kB / 3.32kB 0.7s done
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/./linters/base.py", line 519, in <module>
    _main(args_)
  File "/app/./linters/base.py", line 497, in _main
    lints = _run_linter(file_paths, action_names, action_classes, args)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./linters/base.py", line 402, in _run_linter
    lints_tmp = joblib.Parallel(n_jobs=num_threads, verbose=50)(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 2007, in __call__
    return output if self.return_generator else list(output)
                                                ^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 1650, in _get_outputs
    yield from self._retrieve()
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 1754, in _retrieve
    self._raise_error_fast()
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 1789, in _raise_error_fast
    error_job.get_result(self.timeout)
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 745, in get_result
    return self._return_or_raise()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/joblib/parallel.py", line 763, in _return_or_raise
    raise self._result
RuntimeError: 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
_system failed: cmd='($(find -wholename '*dev_scripts_helpers/documentation/lint_notes.py') -i docs/work_organization/all.contributor_feedback.how_to_guide.md --in_place) 2>&1'
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
truncated output=
09:23:17 - INFO  hdbg.py init_logger:1018                               > cmd='./dev_scripts_helpers/documentation/lint_notes.py -i docs/work_organization/all.contributor_feedback.how_to_guide.md --in_place'
09:23:17 - WARN  hdocker.py build_container_image:387                   Building Docker container...
  ... 2025/04/15 13:23:17 in: []string{}
  ... 2025/04/15 13:23:17 Parsed entitlements: []
  ... #0 building with "default" instance using docker driver
  ... 
  ... #1 [internal] load build definition from Dockerfile
  ... #1 transferring dockerfile: 252B done
  ... #1 DONE 0.0s
  ... 
  ... #2 [internal] load metadata for docker.io/library/node:18
  ... #2 DONE 0.1s
  ... 
  ... #3 [internal] load .dockerignore
  ... #3 transferring context: 2B done
  ... #3 DONE 0.0s
  ... 
  ... #4 [1/3] FROM docker.io/library/node:18@sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e
  ... #4 resolve docker.io/library/node:18@sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e done
  ... #4 sha256:df9fa4e0e39c9b97e30240b5bb1d99bdb861573a82002b2c52ac7d6b8d6d773e 6.41kB / 6.41kB done
  ... #4 sha256:aa6c239d30ee04dede270729f9502389b1a9546687ce656872536340ee0a9e03 2.49kB / 2.49kB done
  ... #4 sha256:de20d623379fc7c7ccf845a22c3153b920d57446ba7c8e64ba25d21a60b48ad6 6.39kB / 6.39kB done
  ... #4 sha256:23b7d26ef1d294256da0d70ce374277b9aab5ca683015073316005cb63d33849 48.49MB / 48.49MB 0.6s done
  ... #4 sha256:07d1b5af933d2dfc3d0dd509d6e20534825e4a537f7b006a6cb5b8e5a1f20905 24.01MB / 24.01MB 0.4s done
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 40.89MB / 64.40MB 0.6s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 17.83MB / 211.33MB 0.6s
  ... #4 sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f 0B / 3.32kB 0.6s
  ... #4 sha256:1eb98adba0eb44a2e4facf9ca3626a4a66feedd0dd56d159cca90a35205744e7 64.40MB / 64.40MB 0.7s
  ... #4 sha256:b617a119f8a27982374d94ec6eb3738ae3d38d6fc2c34c865813926cf596a621 32.51MB / 211.33MB 0.7s
  ... #4 sha256:ee496386c5de1ce84096ca486e1aabcdf7cb8f0afd5a9b4863dbf870b340744f 3.32kB / 3.32kB 0.7s done
Error: Process completed with exit code 1.

@heanhsok do you know if your fix would also fix this?

sonniki avatar Apr 15 '25 13:04 sonniki

Good job @heanhsok

  1. The goal is to push everything through dockerized executables and support sibling and dind indifferently. I am close to being able to do that
  2. I'm ok with restoring the stuff removed (too early).

gpsaggese avatar Apr 15 '25 13:04 gpsaggese

@heanhsok do you know if your fix would also fix this?

Yes it should as it looks like the same error.

heanhsok avatar Apr 15 '25 14:04 heanhsok

  1. I'm fixing this by partial revert, since the PR is not ready yet.

  2. @sonniki I still get this glitching behavior. One line in the golden shows up at different points on different computers. I think it's a race in the OS itself, based on how stderr and stdout are piped together. My suggestion is to filter out that line and / or change the test.

Image

gpsaggese avatar Apr 15 '25 14:04 gpsaggese

@sonniki I still get this glitching behavior. One line in the golden shows up at different points on different computers. I think it's a race in the OS itself, based on how stderr and stdout are piped together. My suggestion is to filter out that line and / or change the test.

Filed #568. Will outsource to an intern to fix.

sonniki avatar Apr 15 '25 16:04 sonniki

Linter GH Action was still throwing an error because of an invalid tmp file path for running prettier. I did a quick fix in #570 to unblock (reverting to the way it was before), LMK if there is a need to follow up

sonniki avatar Apr 15 '25 19:04 sonniki