RF option `--runemptysuite` doesn't work consistently with robot

[MrBIN89]

Description pabot runs incorrect suite in case the --runemptysuite option is used. The robot itself runs zero tests in both cases (with --runemptysuite, without) .

Steps to reproduce:

1. Place two suites from test data (see below) to the directory (check_pabot)
2. run pabot with --runemptysuite option pabot --suite check_pabot.first_suite -e turn_off --runemptysuite .\check_pabot

Expected results: No one test is run, because no matches, though the overall execution result is PASS.

Actual results: The second_suite.robot is executed, though it doesn't match specified tag and suite name

NOTE: pabot executes none tests in case the command line doesn't have --runemptysuite option, but it isn't usefull for me due to the overall execution result FAIL

Suite 'Check Pabot' contains no tests not matching tag 'turn off' in suite 'check_pabot.first_suite'

Test data first_suite.robot:

*** Settings ***
Force Tags    turn_off

*** Test cases ***
Foo Test Case
    Log To Console    Foo Test Case Pass

second_suite.robot:

*** Settings ***
Force Tags    enabled

*** Test cases ***
Bar Test Case
    Log To Console    Bar Test Case Pass

Used versions:

• robotframework-pabot 1.13
• robotframework 4.0.1
• Python 3.6.8

[villesau]

I'm getting Suite 'Notifications' contains no tests or tasks. with following: pabot --processes 4 --exclude failing --outputdir /results --output output1.xml --runemptysuite robot/tests/invitation robot/tests/notifications robot/tests/products so there definitely seems to be something wrong with the command. Yeah, notifications is empty, but that's the exact reason why runemptysuite is used..

[phermann-DGL]

We have the same issue.

pabot --pabotlib --processes 2 --argumentfile1 src/realtime-bonus-engine/configs/de.txt --argumentfile2 src/realtime-bonus-engine/configs/it.txt --report NONE --outputdir "output/realtime-bonus-engine" --logtitle "realtime-bonus-engine log" -x xunitreport.xml --exclude not-ready --exclude no-pipeline --exclude need-assistant --include scheduled --runemptysuite "src/realtime-bonus-engine/tests"

in the testsuite.robot file is no tag scheduled. So there are no tests to execute. We are using this during ci/cd.

used versions:

• robotframework 5.0.1
• robotframework-pabot 2.5.3
• python 3.8.0

[Leemur89]

I am facing the same issue: if all my tests are excluded due to tag filters (I need that for my CI), I need the --runemptyuite option. When used with robot then it is fine, nothing is ran and the return code is 0 When used with pabot, all tests are run When I omit the --runemptysuite option with pabot, I got an error 252 saying that no test matches my filters