reframe icon indicating copy to clipboard operation
reframe copied to clipboard

Issue a more specific error when a fixture of a test fails

Open vkarak opened this issue 3 years ago • 1 comments

Currently, the parent test will simply fail with a DependencyError, whereas we could also point out the failing fixture.

vkarak avatar Oct 29 '21 09:10 vkarak

Here's an example:

class Fix0(rfm.RunOnlyRegressionTest):
    executable = f'echo from fixture Fix0 '

    @sanity_function
    def my_sanity(self):
        return False

@rfm.simple_test
class Base2(rfm.RunOnlyRegressionTest):
    valid_systems = ['dom:login']
    valid_prog_environs = ['builtin']
    executable = 'echo BANANAS'
    f1 = fixture(Fix0, scope='session')

    @sanity_function
    def s(self):
        return True

This fails with:

==============================================================================
SUMMARY OF FAILURES
------------------------------------------------------------------------------
FAILURE INFO for Fix0~dom 
  * Test Description: Fix0~dom
  * System partition: dom:login
  * Environment: builtin
  * Stage directory: /users/jjotero/reframe/stage/dom/login/builtin/Fix0~dom
  * Node list: dom101
  * Job type: local (id=14186)
  * Dependencies (conceptual): []
  * Dependencies (actual): []
  * Maintainers: []
  * Failing phase: sanity
  * Rerun with '-n Fix0~dom -p builtin --system dom:login -r'
  * Reason: sanity error
------------------------------------------------------------------------------
FAILURE INFO for Base2 
  * Test Description: Base2
  * System partition: dom:login
  * Environment: builtin
  * Stage directory: None
  * Node list: 
  * Job type: local (id=None)
  * Dependencies (conceptual): ['Fix0~dom']
  * Dependencies (actual): [('Fix0~dom', 'dom:login', 'builtin')]
  * Maintainers: []
  * Failing phase: startup
  * Rerun with '-n Base2 -p builtin --system dom:login -r'
  * Reason: task dependency error: dependencies failed
------------------------------------------------------------------------------

It would be nice if the failure info for Base would include the f1 attribute name for the fixture.

jjotero avatar Oct 29 '21 09:10 jjotero