Pytest fails to run on a seemingly innocent unit test

[ma-sadeghi]

I have an ongoing PR on the pytest-split repo (a plugin for distributing unit tests) which adds better support for nbval. It basically ensures that all the cells within an IPython notebook belong to one (and only one) test group.

Anyway, I'm stuck writing unit tests for it as it keeps failing for some reason we can't decipher. Here's a minimal example:

def test_dummy1(example_suite):
    result = example_suite.inline_run("--nbval")
    assert len(_passed_test_names(result)) == 5

def test_dummy2(example_suite):
    result = example_suite.inline_run("--nbval")
    assert len(_passed_test_names(result)) == 5

The first test is successful, but the second one strangely fails. I'd really appreciate it if you could look into it and share your insight. Thanks!