robotframework
When merging suites and tests with Rebot, copy documentation, metadata and tags from merged suites and tests
I've attached an example zip file. Inside the merged output.xml file, all the appended suite documentation is still there - but each log contains the complete final text as if it was overwriting the documentation instead of appending to it.
In the sample code I have each testcase appending documentation to the suite, which is lost when the suite results are merged. In real life we append additional test result information to the suite documentation during teardown, so it is conveniently available in the suite summary. Some of the information is the common suite information, but appended to it is additional test summary information (as a table)
However, when a suite is distributed across multiple test servers and we merge the results together, only documentation from one of the servers' results makes it to the final report.
@pekkaklarck suggested I raise this as an issue in this Slack thread: https://robotframework.slack.com/archives/C3C28F9DF/p1653496529741169?thread_ts=1652803426.993159&cid=C3C28F9D
The test environment is: Windows 10 Pro (20H2) python=3.8.13 chromedriver-py==102.0.5005.61 rpaframework==14.0.0 (robotframework 5.0.1) rpaframework-recognition==3.0.0 robotframework-datadriver[XLS]==1.6.1 robotframework-pabot==2.5.3
At the moment when merging suites, we preserve the original suite otherwise but add setup and teardown from the merged suite. It would make sense to copy also suite documentation and metadata. If that's done, then we should also copy documentation and tags when merging tests.
This ought to be pretty easy to implement. Are you @burrk, or someone else, interested to provide a PR? If yes, this could be added to RF 5.1 scope.
I was talking about merging test documentation and tags above, but that didn't make any sense because tests are always replaced fully so docs and tags of the last test are already preserved. This issue was thus only about suite documentation and metadata.
The only design decision that needed to be made was how to handle suite metadata. Alternatives were only using metadata of the merged suite and combining metadata so that items from both are preserved but values in the latter have precedence. In my opinion the latter option made more sense so I decided to go with that one.