TESTAR_dev icon indicating copy to clipboard operation
TESTAR_dev copied to clipboard

Detecting a faulty sequence in the initial state might affect the generation of test sequences

Open ferpasri opened this issue 7 months ago • 0 comments

Internal logic bug

  1. TESTAR obtains the SUT's initial state

When obtaining the SUT's initial state, TESTAR might detect a failure in the SUT and mark the sequence as faultySequence = true

  1. TESTAR executes the begin sequence method

For example, TESTAR performs a login and transitions to a new SUT state

  1. TESTAR obtains the new SUT's state

If the new SUT's state (e.g., after doing a login) does not contain a SUT failure, there is a logic bug that provokes:

  • TESTAR marks the test sequence as faultySequence = true
  • TESTAR marks the state after the begin sequence as Verdict.OK

https://github.com/TESTARtool/TESTAR_dev/blob/9f8d054153f37ab1e86eab199994c7f1a01159af/testar/src/org/testar/monkey/GenerateMode.java#L93-L103

https://github.com/TESTARtool/TESTAR_dev/blob/9f8d054153f37ab1e86eab199994c7f1a01159af/testar/src/org/testar/monkey/DefaultProtocol.java#L780-L808

Steps to reproduce

  • Select the 02_webdriver_parabank protocol
  • Add the regex .*Forgot login info.* as a suspicious tags value
  • Enable the login in the begin sequence

All sequences will be reported as No problem detected.
However, TESTAR will perform the login and will not execute any action.

ferpasri avatar May 10 '25 11:05 ferpasri