`caplog.at_level` affects caplog handler too?

[znicholls]

• [x] a detailed description of the bug or suggestion
• [x] output of pip list from the virtual environment you are using
• [x] pytest and operating system versions
• [x] minimal example if possible

For clarity, I'm not sure if this is a bug or whether we are just using pytest incorrectly.

In https://github.com/pytest-dev/pytest/commit/fcbaab8b0b89abc622dbfb7982cf9bd8c91ef301, the behaviour of at_level was changed so that it affects self.handler too. Was this on purpose? It means that adding the following code to testing/logging/test_fixture.py causes the tests to fail.

# now failing test code
otherlogger = logging.getLogger("tpytest")

def test_with_statement_root_messages_still_logged(caplog):
    with caplog.at_level(logging.INFO):
        with caplog.at_level(logging.CRITICAL, logger=otherlogger.name):
            # doesn't come through because root logger is at INFO level
            logger.debug("root DEBUG level")
            # Is this meant to come through because setting the level of otherlogger
            # should not affect the root logger or have I missed something?
            logger.info("root INFO level")

            # doesn't come through because otherlogger is at CRITICAL level
            otherlogger.warning("otherlogger WARNING level")
            # does come through
            otherlogger.critical("otherlogger CRITICAL level")

    assert "DEBUG" not in caplog.text
    # This test fails. Is the root logger meant to be affected when we change
    # the otherlogger's level?
    assert "INFO" in caplog.text
    assert "WARNING" not in caplog.text
    assert "CRITICAL" in caplog.text

I'm on macOS Mojave 10.14.6, Python 3.7

# pip list output
$ pip list
Package            Version                Location
------------------ ---------------------- ------------------------------------------------
appdirs            1.4.4
argcomplete        1.12.0
attrs              19.3.0
certifi            2020.6.20
cfgv               3.2.0
chardet            3.0.4
distlib            0.3.1
elementpath        2.0.0
filelock           3.0.12
hypothesis         5.25.0
identify           1.4.28
idna               2.10
importlib-metadata 1.7.0
iniconfig          1.0.1
line-profiler      2.1.1
mock               4.0.2
nodeenv            1.4.0
nose               1.3.7
packaging          20.4
pip                20.2.2
pluggy             0.13.1
pre-commit         2.6.0
py                 1.9.0
pyparsing          2.4.7
pytest             6.0.1.dev94+g3f0abcc6a ...Documents/pytest/src
PyYAML             5.3.1
requests           2.24.0
setuptools         49.6.0.post20200814
six                1.15.0
sortedcontainers   2.2.2
toml               0.10.1
tox                3.19.0
urllib3            1.25.10
virtualenv         20.0.30
wheel              0.35.1
xmlschema          1.2.3
zipp               3.1.0