pytest-memray icon indicating copy to clipboard operation
pytest-memray copied to clipboard

Empty report when running with pytest-xdist

Open orsinium opened this issue 3 years ago • 6 comments

Bug Report

Current Behavior The plugin produces an empty output when running tests with pytest-xdist.

Input Code

a = []

def something():
    for i in range(10_000):
        a.append(i)

def test_something():
    something()

Running multiple processes:

image

Running without distribution:

image

Environment

  • Python(s): python3.9, pytest-xdist, pytest-memray

Possible-solutions: For the beginning, it would be sufficient to detect running tests with pytest-xdist and warn the user that -n0 is required to make pytest-memray work. Of course, moving forward, it would be great to make both plugins work together. Running tests in a single process is too hardcore for some projects.

orsinium avatar Apr 21 '22 06:04 orsinium

i am thinking about adding a test and minimal implementation to document the absence of integration between xdist and memray. What is better: return a ExitCode.USAGE_ERROR OR print a warning (warnings.warn kind of warning?) to the terminal and ask the user to pass -n1, if they want to use memray?

petr-tik avatar May 16 '22 18:05 petr-tik

Is there a way we can instead add support for xdist?

gaborbernat avatar May 16 '22 18:05 gaborbernat

Is there a way we can instead add support for xdist?

Yes, but is not trivial. The problem is that currently the information is collected by every worker but the process that is orchestrating the build is the want printing of the summary and when is going done it finds no information because the workers are not sending the information back. Another problem is that we should not leave the result files lying around until the full test suite finishes because it may be a lot of data (currently we delete the files after every test finishes so they never pile up).

pablogsal avatar May 16 '22 18:05 pablogsal

I'm fine with displaying a usage error for now and nudge to -n1. Just would like another issue open that handles support for that.

gaborbernat avatar May 16 '22 18:05 gaborbernat

I'm fine with displaying a usage error for now and nudge to -n1

I will add a test and code for the ExitCode.UsageError then

petr-tik avatar May 16 '22 18:05 petr-tik

I'm fine with displaying a usage error for now and nudge to -n1

A minor correction: it should be -n0. AFAIK, -n1 will run with the distribution but with just one worker. -n0 will disable pytest-xdist altogether.

orsinium avatar May 17 '22 07:05 orsinium