pytest icon indicating copy to clipboard operation
pytest copied to clipboard

Add capteesys capture fixture to bubble up output to `--capture` handler

Open ayjayt opened this issue 4 months ago • 3 comments

potentially closes #12081(and more!)

  • [x] Include documentation when adding new features.
  • [x] Include new tests or update existing tests when applicable.
  • [X] Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.
  • [X] Add text like closes #XYZW to the PR description and/or commits (where XYZW is the issue number). See the github docs for more information.
  • [x] Create a new changelog file in the changelog folder, with a name like <ISSUE NUMBER>.<TYPE>.rst. See changelog/README.rst for details.
  • [x] Add yourself to AUTHORS in alphabetical order.

Explanation

To set global capture behavior, we set --capture=tee-sys so that pytest can a) capture output for printing reports and b) dump output to our terminals. Both of these things are useful.

Passing capture-type fixtures in our tests will block global capture behavior, but allow us to analyze captured output inside the test.

Unfortunately, capture fixtures don't support anything like --capture=tee-sys- until now. This PR adds a new fixture: capteesys which captures output for analysis in tests but also pushes the output to whatever is set by --capture=. It allows the output to "bubble up" to the next capture handler.

Docs build, tests pass.

My Approach (Engineering)

How the system works right now:

Right now, when you include a fixture (syscap, fdcap, etc), that fixture initializes the CaptureFixture wrapper and passes it the class of the implementation it wants to use, one of:

class NoCapture(...): ... # eg. __init__ takes file descriptor #
class SysCapture(...): ... # eg. __init__ takes file descriptor # + several options
class SysCaptureBinary(...): ...
class SysCapture(...): ...
class FDCapture(...): ...
class FDCaptureBinary(...): ...

self.captureclass = ... # whatever was passed

which the wrapper will instantiate as an object later (depending on fixture scope) as such:

def _start(...):
    ...
    out = self.captureclass(1) # captureclass set to one of above classes
    err = self.captureclass(2)

There is no current way to pass other options to the constructor, which would enable other features, such as tee.

Solution:

Discarded Solutions

  • Make that each Capture class accept a tee argument- most will discard it (FDCapture probably could accept it). Most of the time it would be nonsensical and maybe misleading to developers or users.

  • Create a conditional switch for the CaptureFixture wrapper which changes the initialization process based on the actual constructor- But it seems bad to create individual branches in base/containing classes for all possible children/attributes.

Chosen Solution

Since the CaptureFixture class is already initialized by passing the class of the implementation, you just add a new configure dictionary:

capture_fixture = CaptureFixture(SysCapture, request, _ispytest=True)
# to
capture_fixture = CaptureFixture(SysCapture, request, config=dict(tee=True), _ispytest=True)

Which is then expanded when initializing the implementation:

out = self.captureclass(1, **self._config)

It requires very few code changes. This explanation was longer.

To create the capteesys fixture, I just copied the capsys fixture but added the config dictionary as part of its setup.

ayjayt avatar Oct 05 '24 04:10 ayjayt