How can I modify the summary with test results after testing has finished?

[nck974]

I have been able to modify the summary with the following snippet from another issue:

def pytest_html_results_summary(prefix, summary, postfix):
    prefix.extend([
        html.button("Click me", onclick="myFunction()"),
        html.p(id="demo"),
        html.script(raw("""
        function myFunction() {
            document.getElementById('demo').innerHTML = 'Hello World';
        }""")),
    ])

And I am also able to gather the results after the session finishes with:

def pytest_sessionstart(session):
    """
    This hook is called at the beginning and creates a placeholder to store all the results of the
    session
    """
    session.results = dict()

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item):
    """
    Write the results of each test in the session
    """
    outcome = yield

    report = outcome.get_result()
    if report.when == "call":

        # Save result for later processing in pytest_sessionfinish hook
        item.session.results[item] = report

@pytest.hookimpl(tryfirst=True)
def pytest_sessionfinish(session, exitstatus):
    """
    This hook runs after all tests are done
    """
    print()
    print('run status code:', exitstatus)
    passed_amount = sum(1 for result in session.results.values() if result.passed)
    failed_amount = sum(1 for result in session.results.values() if result.failed)
    print(f'there are {passed_amount} passed and {failed_amount} failed tests')
    # TODO: Writing this in a file with a link

My problem is that I can not make a connection to write what I have in session finish with what I have in the summary. The summary hook seems to don't have have access to anyinformation of the session.

Furthermore, after making some prints, it seems the summary is generated before executing the pytest_sessionfinish.

Is there a work around to make my own summary additions after finishing the test?

[nck974]

Well I was able to derive an ugly and hacky solution which I don't think is not the solution to the problem but is enough for me to be able to conitinue.

I was expecting to generate a button in the report that downloads an xml which is formatted as an import for another tool.

I just read the xml file and add at the end my button

@pytest.hookimpl(trylast=True, )
def pytest_sessionfinish(session, exitstatus):
    """
	There seems to be no way of using a hook of pytest-html to provide this summary results, 
	therefore the report is read once created
    and a custom HTML is added with a button which triggers a JS that downloads the XML.
    """
    # Get execution results
    test_cases = {}
    for item, result in session.results.items():

        try:
            tc = re.search(r'^(.+?)_test.py', item.parent.name).group(1)
        except AttributeError:
            logger.error(f'tc not found in module {item.parent.name}')
            raise
        
        # Passed
        if result.passed:
            if tc not in test_cases:
                test_cases[tc] = 'p'
            elif test_cases[tc] == 'f':
                continue
            else:
                test_cases[tc] = 'p'
        
        # Failed
        if result.failed:
            test_cases[tc] = 'f'

    # Generate xml file
	xml = 'x'


    # Get name of the generated report
    output = os.path.join(
        'output',
        'report',
        'my_project',
        datetime.now().strftime("%Y-%m-%d"),
        f'{MY_PROJECT}-test-report.html')

    # Read report
    with open(output) as f:
        lines = f.read()
    
    button = f"""
    <button onclick="downloadBase64File('text/xml','{base64.b64encode(xml.encode('utf8')).decode()}', 'test.xml' )">
    Button to download xml
    </button>
    <span style="color: #756e15;background: #fffbd1;border: 1px solid #87803e;margin: 5px;padding: 5px;" >
        Warning: Add here.
    </span>
    <script>
            function downloadBase64File(contentType, base64Data, fileName) {{
            const linkSource = `data:${{contentType}};base64,${{base64Data}}`;
            const downloadLink = document.createElement("a");
            downloadLink.href = linkSource;
            downloadLink.download = fileName;
            downloadLink.click();
        }}
    </script>
    """
    lines = re.sub(re.escape(
        '</body></html>'),
        f'<div style="padding: 5px;">{button}</div></body></html>',
        lines)
    
    # Save again the report with the same name
    with open(output, 'w') as f:
        f.write(lines)

And looks likes this:

[BeyondEvil]

Not sure what the correct solution is.

But let me label this next-gen, and once we've figured out how to deal with the hooks that modify the report - we can look at this use case.

[BeyondEvil]

Please try 4.0.0rc0 to see if this still is an issue.

[nck974]

I'm not sure if this is an issue. As I commented before I used that hacky solution of writing over the HTML, if a hook can be implemented to be able to what is described in the title would be nice. Otherwise I do not have the need right now to change it.

[BeyondEvil]

I'm not sure if this is an issue. As I commented before I used that hacky solution of writing over the HTML, if a hook can be implemented to be able to what is described in the title would be nice. Otherwise I do not have the need right now to change it.

Not 100% sure I understand the problem.

But here's what the current (4.0.0rc0) implementation looks like:

    @pytest.hookimpl(trylast=True)
    def pytest_sessionfinish(self, session):
        session.config.hook.pytest_html_results_summary(
            prefix=self._report.data["additionalSummary"]["prefix"],
            summary=self._report.data["additionalSummary"]["summary"],
            postfix=self._report.data["additionalSummary"]["postfix"],
        )
        self._report.data["runningState"] = "Finished"
        self._generate_report()

Would changing that to something like:

    @pytest.hookimpl(trylast=True)
    def pytest_sessionfinish(self, session):
        session.config.hook.pytest_html_results_summary(
            prefix=self._report.data["additionalSummary"]["prefix"],
            summary=self._report.data["additionalSummary"]["summary"],
            postfix=self._report.data["additionalSummary"]["postfix"],
            session=session,
        )
        self._report.data["runningState"] = "Finished"
        self._generate_report()

solve your issue?

[nck974]

At first glance I think it would enough. Unfortunately I won't have time to verify that any time soon, as I do not have quick access right now to the code where I was using that.

[BeyondEvil]

At first glance I think it would enough. Unfortunately I won't have time to verify that any time soon, as I do not have quick access right now to the code where I was using that.

No worries. There's no downside afaict to adding the session info to the hook. I'll make it part of the next RC so you can test whenever you have time.

[nahuelmariani]

Hi @BeyondEvil! I'm facing the same situation described by @nck974. I would love to have the session as parameter of the pytest_html_results_summary hook in the new RC. Thank you so much!

[BeyondEvil]

Thanks for reminding me! I’ll see if I can get it added today and release a new RC you can try.