streem
streem copied to clipboard
matz
Cram tests
This isn't perfectly ideal, and I'm not quite done with the tests yet, but it's better than nothing!
:+1: This looks pretty cool, I haven't seen cram before. It would probably help to add a simple make task that runs the cram tests though.
Also, we could probably clean up the syntax a little by using something like here strings and by somehow either adding streem to the user's path or some cram-specific path before the tests are run (or you can just alias streem). I'm not sure that would be possible though, since I haven't used cram myself.
Example
Streem should recognise a stream expression:
$ streem <<< "STDIN | STDOUT"
Syntax OK
I don't like adding dependency in this phase. And I guess that cram doesn't seems to work on windows.
I think cram could be useful, especially if our tests get a little more complicated. Also, we'll probably want to move to a fancier testing tool eventually.
However, we could write our own simple test tool. For example, we could have a script that opens a file with a snippet on each line, inputs each line into streem, and prints out something when there's an error.
Yes, I prefer to make owr test tool.
$ ls t/*.t | bin/streem -e 'STDIN | {|x| system("bin/streem " + x) }'
I just made a very simple test script using @christopherdumas's snippets: https://github.com/nicolasmccurdy/streem/blob/test-snippets/test/test_snippets.sh
However, it's a work in progress, and I should probably hide successes and add some sort of useful status for make and other testing/CI tools.
Cram should work on windows, as long as you have python and pip installed.
I will work on making the tests cleaner like you suggested, @nicolasmccurdy. I will also add tests. If possible, can someone list the features of stream so far so that I can make tests for them?
Windows doesn't support <<< in nicolasmccurdy's comment at least. https://github.com/matz/streem/pull/34#issuecomment-67454503
I didn't notice that, thanks. echo "STDIN | STDOUT" | streem should work instead (assuming that Windows has pipe and echo support).
So are we going to merge this into the main streem, or should I close this??
Not hurry. We know you want to contribute. But this should do carefully considered, I think.
Github's Linguist thinks the test files are Perl files, because the extension is .t. I think it should be changed to .test.
Any reason not to write the tests in C until they can be self-hosted?
Becouse that would be sort of hard and would result in scary test code that no one would want to change, so we would end up here anyway.
Wow, these tests are long... Is that just because they're so detailed, or is there old "good" output of the parser pasted into the tests?
What cram does is it matches the output of the command with it's expected output. That is the output of the parser when run on code. It is long, I agree.
It seems to me that the current tests are either too low level (and too brittle), or too based on existing output of an older version of the code (which is assuming that version has 0 bugs). That being said, cram looks pretty cool, I really liked the older versions of your tests, and I'm not against tests. I'm just not sure if we want something this complicated and brittle, which is based on trusting an older version of what the code did instead of what a human thinks it should be doing directly.
OK. I'll work on using an updated version of streem and making the tests less brittle.