python
Update test command on dev guide landing page
On the landing page, we recommend running: ./python -m test -j3
Since most systems now are multicore, we should consider recommending: ./python -m test -j0 Tests on my machine several minutes faster. We could continue to recommend ./python -m test -j3 for folks using older or single-core machines.
This makes sense.
The linked "Run the tests" gives more info:
If you have a multi-core or multi-CPU machine, you can enable parallel testing using several Python processes so as to speed up things:
./python -m test -j0If you are running a version of Python prior to 3.3 you must specify the number of processes to run simultaneously (e.g.
-j2).
Let's recommend -j0 on the landing page, and we could add more info for older/single-core machines to this longer description.
And it's probably worth removing the "prior to 3.3" by now, 3.2 is EOL since 2016.
@hugovk I would have some mention of older/single core on the landing page since in some parts of the world older/single core machines are the norm. It can refer them to "Run the tests" link for more info.
It is worth updating the GA of CPython too.
https://github.com/python/cpython/blob/613c0d4e866341e15a66704643a6392ce49058ba/.github/workflows/build.yml#L467
Sorry for my ignorance but how does -j0 differ from -j?
From the GNU docs :
If the ‘-j’ option is followed by an integer, this is the number of recipes to execute at once; this is called the number of job slots. If there is nothing looking like an integer after the ‘-j’ option, there is no limit on the number of job slots.
I just wondered why step 3 for build python uses -j and the recommendation for tests is -j0. What's the difference between them?
I'd be happy to have a go at this as a first contribution but just wanted to check my understanding.
There's some inconsistency in the -j options for make and for ./python -m test.
On macOS:
❯ make -j8
Checked 107 modules (31 built-in, 76 shared, 0 n/a on macosx-13.6-arm64, 0 disabled, 0 missing, 0 failed on import)
❯ make -j
Checked 107 modules (31 built-in, 76 shared, 0 n/a on macosx-13.6-arm64, 0 disabled, 0 missing, 0 failed on import)
❯ make -j0
make: the `-j' option requires a positive integral argument
Usage: make [options] [target] ...
Options:
-b, -m Ignored for compatibility.
-B, --always-make Unconditionally make all targets.
-C DIRECTORY, --directory=DIRECTORY
Change to DIRECTORY before doing anything.
-d Print lots of debugging information.
--debug[=FLAGS] Print various types of debugging information.
-e, --environment-overrides
Environment variables override makefiles.
-f FILE, --file=FILE, --makefile=FILE
Read FILE as a makefile.
-h, --help Print this message and exit.
-i, --ignore-errors Ignore errors from commands.
-I DIRECTORY, --include-dir=DIRECTORY
Search DIRECTORY for included makefiles.
-j [N], --jobs[=N] Allow N jobs at once; infinite jobs with no arg.
-k, --keep-going Keep going when some targets can't be made.
-l [N], --load-average[=N], --max-load[=N]
Don't start multiple jobs unless load is below N.
-L, --check-symlink-times Use the latest mtime between symlinks and target.
-n, --just-print, --dry-run, --recon
Don't actually run any commands; just print them.
-o FILE, --old-file=FILE, --assume-old=FILE
Consider FILE to be very old and don't remake it.
-p, --print-data-base Print make's internal database.
-q, --question Run no commands; exit status says if up to date.
-r, --no-builtin-rules Disable the built-in implicit rules.
-R, --no-builtin-variables Disable the built-in variable settings.
-s, --silent, --quiet Don't echo commands.
-S, --no-keep-going, --stop
Turns off -k.
-t, --touch Touch targets instead of remaking them.
-v, --version Print the version number of make and exit.
-w, --print-directory Print the current directory.
--no-print-directory Turn off -w, even if it was turned on implicitly.
-W FILE, --what-if=FILE, --new-file=FILE, --assume-new=FILE
Consider FILE to be infinitely new.
--warn-undefined-variables Warn when an undefined variable is referenced.
-N OPTION, --NeXT-option=OPTION
Turn on value of NeXT OPTION.
This program built for i386-apple-darwin11.3.0
Report bugs to <[email protected]>
And:
❯ ./python.exe -m test -j8
== CPython 3.13.0a0 (heads/main:6ab6040054e5, Oct 5 2023, 08:37:40) [Clang 15.0.0 (clang-1500.0.40.1)]
== macOS-13.6-arm64-arm-64bit little-endian
== Python build: debug
== cwd: /Users/hugo/github/cpython/build/test_python_worker_12564æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
== resources: (all disabled, use -u option)
...
❯ ./python.exe -m test -j0
== CPython 3.13.0a0 (heads/main:6ab6040054e5, Oct 5 2023, 08:37:40) [Clang 15.0.0 (clang-1500.0.40.1)]
== macOS-13.6-arm64-arm-64bit little-endian
== Python build: debug
== cwd: /Users/hugo/github/cpython/build/test_python_worker_12634æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
== resources: (all disabled, use -u option)
...
❯ ./python.exe -m test -j
usage: python -m test [options] [test_name1 [test_name2 ...]]
python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
regrtest.py: error: argument -j/--multiprocess: expected one argument
Pass -h or --help for complete help.
make does not necessarily imply GNU Make; our makefile can also be run with BSD Make. My bsdmake only accepts -j with a positive integer argument.
GNU Make:
-j [jobs], --jobs[=jobs]
Specifies the number of jobs (commands) to run simultaneously. If
there is more than one -j option, the last one is effective. If
the -j option is given without an argument, make will not limit
the number of jobs that can run simultaneously.
BSD Make:
-j max_jobs
Specify the maximum number of jobs that bsdmake may have running
at any one time. Turns compatibility mode off, unless the B flag
is also specified.
So -j0 is the safe option for both these makes and python -m test.
Separately, we can't easily change all these make flavours, but we could make python -m test more accepting and add python -m test -j to do the same as python -m test -j0?
Hey, if the decision has been taken as specified by @willingc I would kindly love to work on this issue meanwhile let me be working on some other issue as I wait for this. Thanks
