cpython icon indicating copy to clipboard operation
cpython copied to clipboard

GH-119169: Speed up `os.walk()`

Open barneygale opened this issue 1 year ago • 1 comments

Handle errors from os.scandir() and ScandirIterator similarly, which lets us loop over directory entries with for. In top-down mode, call os.path.join() at most once per iteration.

$ ./python -m timeit -s "import os" "list(os.walk('.', topdown=True))"
10 loops, best of 5: 37.1 msec per loop  # before
10 loops, best of 5: 35.3 msec per loop  # after
# --> 1.05x faster

$ ./python -m timeit -s "import os" "list(os.walk('.', topdown=False))"
10 loops, best of 5: 31.1 msec per loop  # before
10 loops, best of 5: 29.9 msec per loop  # after
# --> 1.04x faster
  • Issue: gh-119169

barneygale avatar May 19 '24 19:05 barneygale

cc @serhiy-storchaka

vstinner avatar May 20 '24 19:05 vstinner

#119473 fixes the test failure.

barneygale avatar May 23 '24 19:05 barneygale

Closing because reversing sibling traversal order isn't kosher - see discussion on #119473.

barneygale avatar Jul 08 '24 18:07 barneygale