vscode-codeql icon indicating copy to clipboard operation
vscode-codeql copied to clipboard

Properly support "retired" QL tests

Open dbartol opened this issue 4 years ago • 1 comments

Is your feature request related to a problem? Please describe. QL tests in the Test Explorer occasionally go into the "retired" state, where the test result icon is faded. There seems to be little rhyme or reason to when this happens.

Describe the solution you'd like We should properly implement the retire event on the Test Adapter. It was added to Test Explorer after our original implementation, so it seems to use the default behavior of retiring all tests after any reload of the test tree, which we probably do whenever any file in a test directory changes.

Additional context See here for some basic information about how the retire event is supposed to work.

dbartol avatar Oct 27 '20 14:10 dbartol

Going to jot down some notes of a slack discussion we had about this:

  1. We will need to delegate to the cli to determine what the dependencies of each test are. We will need to combine that with a filesystem watcher to retire/re-run any test whose dependencies change. It's unclear where these dependency files should live: a. alongside the tests. One file for each tree. In this case, we need to be careful that we don't accidentally commit these files to VCS. b. in workspace storage. This will keep these new files out of the workspace tree and will be less confusing for users. But, it will be more complex to maintain consistency between them and the tests.
  2. @dbartol also suggested we could store the expand/collapsed state of the test tree across restarts. We could store that inside of the test hierarchy, or in workspace storage. This is a separate, but related feature.

aeisenberg avatar Oct 27 '20 15:10 aeisenberg