[BE][Easy] use `pathlib.Path` instead of `dirname` / `".."` / `pardir`

[XuehaiPan]

Stack from ghstack (oldest at bottom):

• #129409
• -> #129374
• #129426

Changes by apply order:

1. Replace all ".." and os.pardir usage with os.path.dirname(...).

2. Replace nested os.path.dirname(os.path.dirname(...)) call with str(Path(...).parent.parent).

3. Reorder .absolute() ~/ .resolve()~ and .parent: always resolve the path first.

   .parent{...}.absolute() -> .absolute().parent{...}

4. Replace chained .parent x N with .parents[${N - 1}]: the code is easier to read (see 5.)

   .parent.parent.parent.parent -> .parents[3]

5. ~Replace .parents[${N - 1}] with .parents[${N} - 1]: the code is easier to read and does not introduce any runtime overhead.~

   ~.parents[3] -> .parents[4 - 1]~

6. ~Replace .parents[2 - 1] with .parent.parent: because the code is shorter and easier to read.~

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/129374

• :page_facing_up: Preview Python docs built from this PR
• :page_facing_up: Preview C++ docs built from this PR
• :question: Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: No Failures

As of commit 8df444f5c924eae23af3b12923d5e3319896e1ca with merge base 969415885d84b3951d135d84d065586dfe467c05 (): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[XuehaiPan]

Before landing this change, can you please explain why:

@malfet Thanks for the code review. I updated the PR comment.

---

• You sometime replace parents[1] with .parent.parent

I replace .parents[1] with .parents[2 - 1], while I think .parent.parent is shorter and easier to understand.

---

• Why parents[x - 1] notation?

.parents[x - 1] is equal to chain .parent for x times. I think .parents[4 - 1] is better than .parents[3] to represent .parent.parent.parent.parent. Also these is no runtime overhead for .parents[x - 1] because x - 1 is a compile time constant.

$ echo 'Path("torch").resolve().parent[4 - 1]' | python3 -m dis
  0           0 RESUME                   0

  1           2 PUSH_NULL
              4 LOAD_NAME                0 (Path)
              6 LOAD_CONST               0 ('torch')
              8 CALL                     1
             16 LOAD_ATTR                3 (NULL|self + resolve)
             36 CALL                     0
             44 LOAD_ATTR                4 (parent)
             64 LOAD_CONST               1 (3)
             66 BINARY_SUBSCR
             70 POP_TOP
             72 RETURN_CONST             2 (None)

---

• Path("/").parents[0] will throw an exception, while Path("/").parent is always safe to call, should this be a considered in some of the cases?

No, the most shallow directory is the PyTorch repo root. It is safe to use .parents[...].

[XuehaiPan]

I opened another PR to replace .resolve() with .absolute().

• #129409

This PR only contains semantic equivalent changes and the PR comment is also updated.

[XuehaiPan]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here

[pytorchmergebot]

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

• BC Lint / bc_linter

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

[XuehaiPan]

@pytorchbot merge -i

[pytorchmergebot]

Merge started

Your change will be merged while ignoring the following 1 checks: BC Lint / bc_linter

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here

[pytorchmergebot]

Merge failed

Reason: 1 jobs have failed, first few of them are: inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_huggingface, 1, 1, linux.g5.4xlarge.nvidia.gpu)

Details for Dev Infra team

Raised by workflow job

[XuehaiPan]

@pytorchbot merge -f "ci failure unrelated"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here

[huydhn]

@pytorchbot revert -m 'Sorry for reverting your change but it causes lots of internal build failures where they fail to find hipify module' -c ghfirst

Here is an example trace FYI:

  File "/re_cwd/buck-out/v2/gen/fbcode/cd9e8a7f2c0fa73f/caffe2/tools/amd_build/__build_amd__/build_amd#link-tree/__run_lpar_main__.py", line 73, in <module>
    __invoke_main()
  File "/re_cwd/buck-out/v2/gen/fbcode/cd9e8a7f2c0fa73f/caffe2/tools/amd_build/__build_amd__/build_amd#link-tree/__run_lpar_main__.py", line 35, in __invoke_main
    run_as_main(module, main_function)
  File "/re_cwd/buck-out/v2/gen/fbcode/cd9e8a7f2c0fa73f/caffe2/tools/amd_build/__build_amd__/build_amd#link-tree/__par__/meta_only/bootstrap.py", line 98, in run_as_main
    oss_run_as_main(
  File "/re_cwd/buck-out/v2/gen/fbcode/cd9e8a7f2c0fa73f/caffe2/tools/amd_build/__build_amd__/build_amd#link-tree/__par__/bootstrap.py", line 69, in run_as_main
    runpy._run_module_as_main(main_module, alter_argv=False)
  File "/usr/local/fbcode/platform010/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/fbcode/platform010/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/re_cwd/buck-out/v2/gen/fbcode/cd9e8a7f2c0fa73f/caffe2/tools/amd_build/__build_amd__/build_amd#link-tree/build_amd.py", line 13, in <module>
    from hipify import hipify_python  # type: ignore[import]
ModuleNotFoundError: No module named 'hipify'

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here. Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

Reverting PR 129374 failed

Reason: Command git -C /home/runner/work/pytorch/pytorch revert --no-edit 0314c4c101c44d5d89b4fad9d37a012dc6f31128 returned non-zero exit code 1

CONFLICT (modify/delete): benchmarks/dynamo/ci_expected_accuracy/cu124/update_expected.py deleted in HEAD and modified in parent of 0314c4c101 ([BE][Easy] use `pathlib.Path` instead of `dirname` / `".."` / `pardir` (#129374)).  Version parent of 0314c4c101 ([BE][Easy] use `pathlib.Path` instead of `dirname` / `".."` / `pardir` (#129374)) of benchmarks/dynamo/ci_expected_accuracy/cu124/update_expected.py left in tree.
Auto-merging docs/source/scripts/exportdb/generate_example_rst.py
Auto-merging torch/_inductor/runtime/compile_tasks.py
error: could not revert 0314c4c101... [BE][Easy] use `pathlib.Path` instead of `dirname` / `".."` / `pardir` (#129374)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git revert --continue".
hint: You can instead skip this commit with "git revert --skip".
hint: To abort and get back to the state before "git revert",
hint: run "git revert --abort".
hint: Disable this message with "git config advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

[huydhn]

@pytorchbot revert -m 'Sorry for reverting your change but it causes lots of internal build failures where they fail to find hipify module' -c ghfirst

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here. Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

@XuehaiPan your PR has been successfully reverted.

[XuehaiPan]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here

[huydhn]

@pytorchbot revert -m 'Sorry for reverting your change but it is still failing with the same error' -c ghfist

ModuleNotFoundError: No module named 'hipify', let me take a closer look next week into the error.

[pytorch-bot[bot]]

❌ 🤖 pytorchbot command failed:

@pytorchbot revert: error: argument -c/--classification: invalid choice: 'ghfist' (choose from 'nosignal', 'ignoredsignal', 'landrace', 'weird', 'ghfirst')

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Try @pytorchbot --help for more info.

[huydhn]

@pytorchbot revert -m 'Sorry for reverting your change but it is still failing with the same error' -c ghfirst

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here. Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

Reverting PR 129374 failed

Reason: Command git -C /home/runner/work/pytorch/pytorch revert --no-edit 9e1f3ecaa710785a1ab03c6ad5093a5566d6c5e5 returned non-zero exit code 1

Auto-merging tools/code_coverage/package/oss/utils.py
Auto-merging tools/code_coverage/package/util/setting.py
Auto-merging tools/gen_vulkan_spv.py
CONFLICT (content): Merge conflict in tools/gen_vulkan_spv.py
Auto-merging tools/setup_helpers/cmake.py
Auto-merging tools/setup_helpers/generate_code.py
CONFLICT (content): Merge conflict in tools/setup_helpers/generate_code.py
Auto-merging tools/stats/import_test_stats.py
CONFLICT (content): Merge conflict in tools/stats/import_test_stats.py
Auto-merging tools/test/heuristics/test_heuristics.py
CONFLICT (content): Merge conflict in tools/test/heuristics/test_heuristics.py
Auto-merging tools/test/heuristics/test_interface.py
CONFLICT (content): Merge conflict in tools/test/heuristics/test_interface.py
Auto-merging tools/test/heuristics/test_utils.py
CONFLICT (content): Merge conflict in tools/test/heuristics/test_utils.py
Auto-merging tools/test/test_gen_backend_stubs.py
CONFLICT (content): Merge conflict in tools/test/test_gen_backend_stubs.py
Auto-merging tools/test/test_test_selections.py
CONFLICT (content): Merge conflict in tools/test/test_test_selections.py
Auto-merging tools/test/test_upload_stats_lib.py
CONFLICT (content): Merge conflict in tools/test/test_upload_stats_lib.py
Auto-merging tools/testing/discover_tests.py
Auto-merging tools/testing/explicit_ci_jobs.py
CONFLICT (content): Merge conflict in tools/testing/explicit_ci_jobs.py
Auto-merging tools/testing/modulefinder_determinator.py
CONFLICT (content): Merge conflict in tools/testing/modulefinder_determinator.py
Auto-merging tools/testing/target_determination/gen_artifact.py
CONFLICT (content): Merge conflict in tools/testing/target_determination/gen_artifact.py
Auto-merging tools/testing/target_determination/heuristics/filepath.py
Auto-merging tools/testing/target_determination/heuristics/llm.py
Auto-merging tools/testing/target_determination/heuristics/previously_failed_in_pr.py
Auto-merging tools/testing/target_determination/heuristics/utils.py
Auto-merging tools/testing/test_selections.py
CONFLICT (content): Merge conflict in tools/testing/test_selections.py
Auto-merging torchgen/gen_backend_stubs.py
CONFLICT (content): Merge conflict in torchgen/gen_backend_stubs.py
Auto-merging torchgen/gen_lazy_tensor.py
CONFLICT (content): Merge conflict in torchgen/gen_lazy_tensor.py
error: could not revert 9e1f3ecaa7... [BE][Easy] use `pathlib.Path` instead of `dirname` / `".."` / `pardir` (#129374)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git revert --continue".
hint: You can instead skip this commit with "git revert --skip".
hint: To abort and get back to the state before "git revert",
hint: run "git revert --abort".
hint: Disable this message with "git config advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

[huydhn]

@pytorchbot revert -m 'Sorry for reverting your change but it is still failing with the same error' -c ghfirst

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here. Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

@XuehaiPan your PR has been successfully reverted.

[github-actions[bot]]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[XuehaiPan]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here