psutil is missing in setup dependencies

[pzloty]

When running from a virtual environment shell on Windows 10 (2004): pipenv run python speed_test.py

I got this warning: WARNING: unable to increase process priority for each process and one extra for each runner.timeit() call.

Figured out that it is because psutil is missing in package dependencies.

[vstinner]

I got this warning: WARNING: unable to increase process priority for each process and one extra for each runner.timeit() call.

The minimum work which can be done is to enhance the error message to mention that psutil is recommended on Windows.

Something like:

        if set_cpu_affinity(cpus):
            if self.args.verbose:
                if isolated:
                    text = ("Pin process to isolated CPUs: %s"
                            % format_cpu_list(cpus))
                else:
                    text = ("Pin process to CPUs: %s"
                            % format_cpu_list(cpus))
                print(text)

            if isolated:
                self.args.affinity = format_cpu_list(cpus)
        else:
            if not isolated:
                print("ERROR: CPU affinity not available.", file=sys.stderr)
                print("Use Python 3.3 or newer, or install psutil dependency")
                sys.exit(1)
            elif not self.args.quiet:
                print("WARNING: unable to pin worker processes to "
                      "isolated CPUs, CPU affinity not available")
                print("Use Python 3.3 or newer, or install psutil dependency")

Would you be interested to propose a PR for that?

---

psutil is mostly useful on Windows. But I didn't feel brave enough to use environment markers in setup.py.

pyperf used environment markers when Python 2.7 was still supported:

diff --git a/setup.py b/setup.py
index fd5207f..3c85bd4 100644
--- a/setup.py
+++ b/setup.py
@@ -59,7 +59,12 @@ def main():
         'author_email': '[email protected]',
         'classifiers': CLASSIFIERS,
         'packages': ['perf', 'perf.tests'],
-        'install_requires': ["statistics; python_version < '3.4'", "six"],
+        'install_requires': ["six"],
+        # don't use environment markers in install_requires, but use weird
+        # syntax of extras_require, to support setuptools 18
+        'extras_require': {
+            ":python_version < '3.4'": ["statistics"],
+        },
         'entry_points': {
             'console_scripts': ['pyperf=perf.__main__:main']
         }

which caused issues with old setuptools versions: https://pyperf.readthedocs.io/en/latest/run_benchmark.html#install-pyperf

I'm open to reintroduce environment markers to require psutil, but only on Windows.

Another option is to write a C extension just for set_cpu_affinity().

Another option is to simply make the warning quiet on Windows.

[pzloty]

Yes, adding another line to the warning would be sufficient with minimal code.:

def _process_priority(self):
    if not MS_WINDOWS:
        return
    if not set_highest_priority():
        print("WARNING: unable to increase process priority")
        print("Use Python 3.3 or newer, or install psutil dependency")  # <---

My first choice was:

install_requires=[
        "psutil>=5.7;platform_system=='Windows'"
    ]

But I understand that it's not required for basic functionality.

I got the warning because of the set_highest_priority().

Suppressing warning/error/exception leads to confusion. It would be much nicer to display a warning as soon as psutil import fails. E.g. by wrapping it:

Replacing all import psutil with:

import pyperf._psutil as psutil

And then in _psutil.py file:

try:
    # Optional dependency
    from psutil import *
except ImportError:
    __IMPORT_FAILED = True
    raise ImportError("psutil")
else:
    __IMPORT_FAILED = False
finally:
    if __IMPORT_FAILED:
        print("Optional dependency 'psutil' is not available: some features will not be available.")

And then in set_highest_priority() and set_cpu_affinity() check psutil against None instead of trying to reimport it. Is there any reason for that reimport? But I'm not sure if it has any negative consequences.

[vstinner]

install_requires=["psutil>=5.7;platform_system=='Windows'"]

If you propose a PR to implement that, I can test it ;-)

[gvanrossum]

I get this too with 3.11 built from main, today. It seems psutil is not compatible with the latest main branch. Though it looks like psutil still builds with 3.11a6, and then the set_highest_priority() function (in _cpu_utils.py) returns True, so you wouldn't get the error.