nbmake icon indicating copy to clipboard operation
nbmake copied to clipboard

Notebook tests don't have names in pytest junit reports

Open hulsed opened this issue 1 year ago • 0 comments

Describe the bug Notebook tests are not being given names in junit reports. This makes it difficult to differentiate different tests in CI tools like bamboo that parse junit.xml to determine test names.

To Reproduce Steps to reproduce the behavior:

  1. From any repo, such as nbmake-examples, run nbmake from pytest and have it generate a junit report, e.g., "pytest --junit-xml=junit.xml --nbmake"
  2. Open the junit report for a notebook test and look at the "name" property for each test, which will be blank.

Expected behavior Names should be populated with something similar to "classname".

Screenshots Notebook tests (no names) junit output from nbmake-examples (notice the lack of names given for each test): image

Example of what doctests produce for names: image

How nameless tests show up in bamboo (no info on which test this was): image

Desktop (please complete the following information):

  • OS: Windows and Linux

Additional context I believe a very easy fix for this would be to simply pass a name from the NotebookItem class during initialization, by changing pytest_items.py from:

  class NotebookItem(pytest.Item):
      nbmake = True
  
      def __init__(self, "", filename: str):
          super().__init__("Notebook: "+parent, parent)
          self.filename = filename

To:

  class NotebookItem(pytest.Item):
      nbmake = True
  
      def __init__(self, parent: Any, filename: str):
          super().__init__("Notebook: "+parent, parent)
          self.filename = filename

hulsed avatar Oct 24 '24 21:10 hulsed