acton
acton copied to clipboard
`testing` improvements
- [x] rename
unit_test_runnertotest_runner(and vice versa since there is already a test_runner, which actually just runs unit_tests) - [x] remove testing.AssertError and replace its use with AssertionError (builtin)
- [x] send
Loggerto test function - [x] add ActorTests and EnvTests classes + add to args in `test_runner(env, unit_tests, actor_tests, env_test)
- [x] actually implement SyncActorTests
- [x] actually implement AsyncActorTests
- [x] actually implement EnvTests
- [x] #1631
- [x] implement
__test_maindiscovery in compiler -> produce.test_root.c - [x] add builder support for
.test_root.c - [x] depends on #1427
- [x] add command to just list all tests
- [x] add option to run a single test
- [x] separate printing status from keeping results up to date etc
- printing should be a separate actor that receives status updates
- the current test runner should instantiate such an actor and send all the updates to it
- [x] add support for JSON output
- building on the previous item of separating printing
- now becomes possible to run multiple test programs that all output
- [x] do multiple test loops (per default) to verify stability and non-flakiness
- like 10 loops? or maybe try to run for 0.1 seconds or something?
- [x] support progress reporting, not just final status
- useful for multiple iterations in test loop
- [ ] build special logger that can capture output and display on error?
- [ ] maybe add support for golden tests?
- [ ] perf timing history
- [x] record snapshot to a file on disk
acton test --record - [ ] perf_data file should include history of perf snapshots, not just latest one, this way we can draw a graph over time
- [x]
acton testshould read latest snapshot and display perf results including comparison - [ ] time module to have humanize() function to display time in human terms, like 123ns or 123.4ms, so pick suitable prefix and format so we get like 4 characters or something compact like that
- [ ]
12ms +1 14ms -1ms 16ms -1ms
- [x] record snapshot to a file on disk