rascal icon indicating copy to clipboard operation
rascal copied to clipboard

beginsAfter test fails once more (randomly)

Open jurgenvinju opened this issue 3 years ago • 7 comments

Describe the bug

Error: [ERROR] Tests run: 5798, Failures: 0, Errors: 1, Skipped: 198, Time elapsed: 452.947 s <<< FAILURE! - in org.rascalmpl.test.AllSuiteParallel
Error: [ERROR] beginsAfter1: <13508,117>(org.rascalmpl.test.infrastructure.RascalJUnitParallelRecursiveTestRunner$ModuleTester)  Time elapsed: 0.006 s  <<< ERROR!
java.lang.Exception: 
Test lang::rascal::tests::basic::Locations::beginsAfter1 failed due to
	test returned false

Actual parameters:
	int =>0

Taken from: https://github.com/usethesource/rascal/actions/runs/3345467043/attempts/1

This was an action run for a release tag that failed due to this function.

jurgenvinju avatar Oct 28 '22 12:10 jurgenvinju

Error: [ERROR] Tests run: 5817, Failures: 0, Errors: 1, Skipped: 197, Time elapsed: 189.707 s <<< FAILURE! - in org.rascalmpl.test.AllSuiteParallel
Error: [ERROR] beginsAfter1: <13508,117>(org.rascalmpl.test.infrastructure.RascalJUnitParallelRecursiveTestRunner$ModuleTester)  Time elapsed: 0 s  <<< ERROR!
java.lang.Exception: 
Test lang::rascal::tests::basic::Locations::beginsAfter1 failed due to
	test returned false

Actual parameters:
	int =>-1559710758

Failed during https://github.com/usethesource/rascal/actions/runs/4733580658

jurgenvinju avatar Apr 19 '23 08:04 jurgenvinju

@PaulKlint if you have time, could you have a look? This failed a release accidentally again.

jurgenvinju avatar Apr 19 '23 08:04 jurgenvinju

Here's another one (during a release): https://github.com/usethesource/rascal/actions/runs/5003114296/jobs/8964006146

jurgenvinju avatar May 17 '23 14:05 jurgenvinju

Error: [ERROR] Tests run: 5849, Failures: 0, Errors: 1, Skipped: 199, Time elapsed: 168.441 s <<< FAILURE! - in org.rascalmpl.test.AllSuiteParallel
Error: [ERROR] beginsAfter1: <13508,[117](https://github.com/usethesource/rascal/actions/runs/5185932055/jobs/9346472945#step:9:118)>(org.rascalmpl.test.infrastructure.RascalJUnitParallelRecursiveTestRunner$ModuleTester)  Time elapsed: 0.004 s  <<< ERROR!
java.lang.Exception: 
Test lang::rascal::tests::basic::Locations::beginsAfter1 failed due to
	test returned false

Actual parameters:
	int =>6

here we go again.

jurgenvinju avatar Jun 06 '23 10:06 jurgenvinju

again: https://github.com/usethesource/rascal/actions/runs/5344293811

jurgenvinju avatar Jun 22 '23 11:06 jurgenvinju

I have @ignore'd this test because it seems to always pop-up while I'm doing complex release bootstraps and it never has to do anything with what I'm doing.

@PaulKlint Let's pick this up again when we start un-ignoring more tests after the first compiler bootstrap?

jurgenvinju avatar Jun 22 '23 14:06 jurgenvinju

Agreed, I have already spent hours on these tests and it is extremely hard to get them 100% right. Let's turn them off and switch to another testing strategy, e.g., instead of randomly generated loc, we could use fixed predefined constant ones.

PaulKlint avatar Jun 22 '23 22:06 PaulKlint