tern icon indicating copy to clipboard operation
tern copied to clipboard

Overhaul Tern's continuous integration testing

Open rnjudge opened this issue 3 years ago • 1 comments

Describe the Feature Currently Tern uses ci/test_files_touched.py to run various Tern tests based on which files were changed in a commit. This way of testing has proved to be porous, however, and many regressions have not been caught. Examples of errors that have been introduced with new commits but not caught with the test_files_touched scripts:

  • Output report stops collecting packages for certain package managers (#924 )
  • Output report changes -- i.e. adding duplicate notices to the report (#927)
  • Loading from the cache causes unexpected output report content (#899)

Use Cases This will be helpful to catch bugs before they are introduced to the code base instead of after the fact.

Implementation Changes We should test all of the code no matter what files were changed. We should also be verifying the contents of the reports, not just making sure that the reports are generated.

Related Issues*

  • #914

rnjudge avatar Mar 21 '21 20:03 rnjudge

@rnjudge would this be a good issue to complete for the Beta release?

nishakm avatar Jan 26 '22 16:01 nishakm