reporter icon indicating copy to clipboard operation
reporter copied to clipboard

Move test cases to Detached section which are not in the test report anymore

Open chura90 opened this issue 1 year ago • 7 comments
trafficstars

In case the title test of the case is changed during the next report importing there will be two test cases old one and new with title changed

We can automatically move these test cases to the Detached section if these test cases are not in the test report anymore

Risk and Challenge we have several file imports from mobile/admin/core projects so how to detect from which file the test comes from Can we determine this based on the report file name?

chura90 avatar May 10 '24 10:05 chura90

I guess it will be more reliable to detect changes(additions/deletions/updates) by resolving a test suite name,id rather then test run Can you provide a command you use to send report in order to give a little bit more of context?

Thanks

tikolakin avatar May 10 '24 12:05 tikolakin

ci script: - apt-get install -y xmlstarlet - xmlstarlet ed --inplace --delete "//testsuite/system-out" ./TESTS.xml - xmlstarlet ed --inplace --delete "//testsuite/system-err" ./TESTS.xml - xmlstarlet ed --inplace --update "//testsuite[contains(@name, 'Chrome Headless')]/@name" --value "@Sadmin" ./TESTS.xml - npm i -g @testomatio/reporter - npx report-xml "./TESTS.xml" --lang="other"

so different projects import their own reports
Screenshot 2024-05-10 at 15 34 02

so if we import tests to Mobile the Core test should not be marked as detached

chura90 avatar May 10 '24 12:05 chura90

Thanks, We have different suites as well, yet slightly different problems with detach state. Basically, we don't use detach option at all when sending test to testomatio. (note, we send tests to testomatio before test execution)

I guess, a simple fix for you would be to merge all your XML reports (Core, Admin, Mobile) into a single one and send it once.

tikolakin avatar May 10 '24 12:05 tikolakin

Just to give you an idea on how no detach state is implemented in testomatio for other reporting types https://github.com/testomatio/check-tests?tab=readme-ov-file#disable-detached-tests

tikolakin avatar May 10 '24 12:05 tikolakin

Hm, for now we are sending 3 xml reports at least, but there are no tests are marked as detached I mean we have 3 report with different test cases And the last question if merging all reports into a single one will be enough performance for Testomat to analyze more than 3000 tests at one report?

chura90 avatar May 14 '24 11:05 chura90

It should be fairly easy to test it out, generally, 3k should not impact performance

but there are no tests are marked as detached I mean we have 3 report with different test cases

so you want test to be marked as detached when existing test case in testomatio is not present in a latest import?

tikolakin avatar May 14 '24 11:05 tikolakin

yes...when the title of test case is changed so the new one is imported and the old one should be moved to detached when the developer changes old test(rename) during the next import this test is imported but old one should be as detached

chura90 avatar May 14 '24 11:05 chura90

Implemented in https://github.com/testomatio/app/issues/1087

DavertMik avatar Dec 18 '24 21:12 DavertMik

closing as implemented

TetianaKhomenko avatar Jan 17 '25 09:01 TetianaKhomenko