noisepage icon indicating copy to clipboard operation
noisepage copied to clipboard

Published 20 hours ago • verified publisher icon cmu-db

JUnit Test Dry Runs Not Supported

Open turingcompl33t opened this issue 4 years ago • 3 comments

All of the internals of JUnit tests (i.e. the infrastructure like TestServer, NoisePageServer, etc.) appear to support JUnit test dry runs in which the executed commands are printed to the terminal but the DBMS is never actually started and the tests are not run. This is a useful debugging feature. However, currently, this functionality is not exposed outwardly by the top-level JUnit test running code - passing the argument --dry-run (the string expected by the lower-level logic in the constructor for TestServer instances) to the JUnit script results in a runtime exception because the argument is not recognized by the argument parsing logic.

Furthermore, there appears to be at least one logic bug regarding JUnit dry runs in which a runtime exception is thrown by the testing infrastructure when the system is actually in a consistent state. The one I am aware of occurs in NoisePageServer::stop_db() because of faulty logic in the check for a test dry run, but there may be other cases like this that are simply not being exercised at present.

The fix for these issues should be relatively straightforward.

turingcompl33t avatar Feb 13 '21 16:02 turingcompl33t

Whoops. I probably broke these.

lmwnshn avatar Feb 13 '21 16:02 lmwnshn

I should probably fix this at some point, but I want to note that the general approach of "print out commands before you run them" is pretty decent. I don't really trust dry-run code to stay in sync.

lmwnshn avatar May 24 '21 23:05 lmwnshn

Agree with the comment above. Can't really recall now what exactly I was working on when I opened this issue. I think the main reason I did so was because there are often times when I am working on one of the dev machines that I can't run the integration tests because the DBMS port is already in use. Maybe a more useful (and simpler) solution would be to just allow configuration of the DBMS port for the tests themselves (or at least this would solve my particular issue).

Obviously we can already pass arbitrary arguments to the DBMS, but I think that 15721 is hard-coded into the test infrastructure at a couple points? Would need to take a closer look.

turingcompl33t avatar May 25 '21 14:05 turingcompl33t