forth2012-test-suite icon indicating copy to clipboard operation
forth2012-test-suite copied to clipboard

Configuration file

Open gerryjackson opened this issue 9 years ago • 4 comments

I've created a new release v0.13.1 with the recent changes we have made and also uploaded a new test file called prelimtest.fth (see prelimtest.md for details) into the master branch.

In the next release I think we ought to include the configuration file we have agreed would be desirable. The question is what do we want to put into it, items off the top of my head are:

  • Specify which tests to carry out
  • Blocks, block numbers to be used in the test and characters per line in a block
  • Non-existent word for FIND SEARCH-LIST to fail with (guaranteed to not be in the dictionary)
  • Word for NAME>INTERPRET to fail with ( or none if applicable )
  • Directory path for the tests

Anything I've forgotten and any other suggestions?

gerryjackson avatar Jan 19 '16 20:01 gerryjackson

Whether >FLOAT allows leading or trailing spaces? (See #16)

steverpalmer avatar Jan 20 '16 10:01 steverpalmer

A few other candidates:

  • BITS/CELL - currently derived in utilities, but maybe the tester should be given the opportunity of defining it rather than rely on a derived value.
  • Whether to allow interactive tests that interrupt the full test run (ACCEPT).
  • VERBOSE could be moved from (t)tester to config

steverpalmer avatar Jan 20 '16 10:01 steverpalmer

Selection of tester or ttester.

steverpalmer avatar Jan 21 '16 10:01 steverpalmer

Thanks for the suggestions Steve, to collect together the list in no particular order:

  • Specify which tests to carry out
  • Blocks, block numbers to be used in the test and characters per line in a block
  • Non-existent word for FIND SEARCH-LIST to fail with (guaranteed to not be in the dictionary)
  • Word for NAME>INTERPRET to fail with ( or none if applicable )
  • Directory path for the tests
  • Selection of tester or ttester
  • BITS/CELL - currently derived in utilities, but maybe the tester should be given the opportunity of defining it rather than rely on a derived value.
  • Whether to allow interactive tests that interrupt the full test run (ACCEPT).
  • VERBOSE could be moved from (t)tester to config
  • Whether >FLOAT allows leading or trailing spaces? (See #16)

It would be nice if the config file could only use the subset of words tested in prelimtest.fth as we could then insert it before the first core test. I'm tied up this weekend and will have a go at it after. I'm not sure of the best way to select which tests to carry out, up to now I've just relied on commenting out the lines that include the test files. Perhaps a set of variables that are on or off e.g. variable core-wordset variable core-ext-wordset ... followed by core-wordset off core-ext-wordset on etc It's where a GUI with tick boxes would come in handy.

gerryjackson avatar Jan 22 '16 17:01 gerryjackson