grails-cucumber icon indicating copy to clipboard operation
grails-cucumber copied to clipboard

output from test reports doesn't seem right

Open ppazos opened this issue 6 years ago • 4 comments

Hi I'm testing this plugin on Grails 2.5.6 (the latest 2.x version), and I have this feature file:

Feature: xyz
  some description here

  Scenario: aaa
    Given ...
    When ...
    Then ...

  Scenario: bbb
    Given ...
    When ...
    Then ...

  Scenario: ccc
    Given ...
    When ...
    Then ...

On grails test-app :cucumber I the output for Scenario aaa in the test html report is "Feature xyz ...", then for Scenario bbb the output is "Scenario aaa ....", and for Scenario ccc the output is "Scenario bbb .... Scenario ccc".

It seems the outputs are not mapped to the correspondent Scenario being executed.

ppazos avatar Dec 14 '18 04:12 ppazos

uhh, looks like the reporting in cucumber or grails has changed so that the current mapping from cucumber to junit/grails doesn't fit anymore. See CucumberTestType, you could use the DebugFormatter here to find out where it is going wrong.

hauner avatar Dec 16 '18 15:12 hauner

Thanks @hauner do you know how can I include the modified plugin on an app to test this?

ppazos avatar Dec 20 '18 19:12 ppazos

Got it working adding this to the buildconfig grails.plugin.location.cucumber = '/home/pablo/GitHub/grails-cucumber'

I'm seeing everything that outputs with R is empty, only the F steps have some info:

F(scenario): (Upload an ADL 1.4-based OPT)
F(step): (current count of OPTs is 0)
F(step): (the client uploads an OPT to the I_DEFINITION_ADL14 with template_id "encounter")
F(step): (current count of OPTs is 1)
R(match)
R(result):
R(match)
R(result):
R(match)
R(result):
R(after)
R(after)
AFTER2
R(after)
F(endOfScenarioLifeCycle): (Upload an ADL 1.4-based OPT)
F(startOfScenarioLifeCycle): (Get template by template_id)
R(before)

Also seeing the behavior described in https://github.com/hauner/grails-cucumber/issues/52 all the Before and After methods are called on every scenario, even if those are declared on different step files.

But can't see what is going on in the Cucumber Formatter with the report of the features and scenarios, there is too much IoC there to understand the workflow.

One thing I found is the activeFeature = null was missing from here:

private void endFeature () {
        if (activeFeature) {
            log.trace ("end feature ()\n")

            report.endFeature ()
            publisher.testCaseEnd (activeFeature.name)
            activeFeature = null
        }
    }

Not sure if that fixed anything. On the HTML report I get the output of the feature and first scenario on the second scenario, and the first scenario has no output. Also getting the output of the second and third scenarios on the third scenario.

I'm not sure how the HTML report is created and how to associate the specific outputs with the correct scenario. Also not sure if the feature output should be there at all, since there is no row for the feature, just seeing rows for the scenarios. It is easier with an image:

unit test results all tests

ppazos avatar Dec 20 '18 20:12 ppazos

I Looked a minute at the code, the CucumberFormatter is the one that tries to translate from cucumber to junit and feeds the GrailsTestEventPublisher that is probably the source of the report generated by grails.

hauner avatar Feb 10 '19 13:02 hauner