tap4j icon indicating copy to clipboard operation
tap4j copied to clipboard

Published 20 hours ago • verified publisher icon tupilabs

Create a PEG parser

Open kinow opened this issue 3 years ago • 8 comments

The regex parser has grown to be somewhat complicated, so let's try a PEG parser and see if that'd be easier to maintain.

  • [x] TestSet
  • [x] Plan
  • [x] TestResult
  • [x] Comments
  • [x] Empty Plan
  • [x] Directives
  • [x] Yaml Block
  • [x] Empty lines
  • [ ] Subtests

Subtests are not in the TAP13 specification, but are used by Perl tools, and Jenkins TAP plug-in supports it too. Ideally the new parser ought to support it too.

kinow avatar Nov 11 '21 11:11 kinow

Current progress, parsing all existing TAP steam in the test-resources folder with the new PEG parser:

34.62% - failed [34] success [18] total [52]

kinow avatar Dec 03 '21 04:12 kinow

53.85% - failed [24] success [28] total [52]

kinow avatar Dec 03 '21 06:12 kinow

Maybe helpful for the indent/dedent part: https://old.reddit.com/r/ProgrammingLanguages/comments/si0fvd/how_to_parse_indentationbased_syntax_with_parser/

kinow avatar Feb 01 '22 22:02 kinow 

[ERROR] Tests run: 167, Failures: 0, Errors: 1, Skipped: 0

[ERROR] Errors: 
[ERROR] com.tupilabs.tap4j.AllFilesTest.dynamicTests
[INFO]   Run 1: PASS
[INFO]   Run 2: PASS
[INFO]   Run 3: PASS
[INFO]   Run 4: PASS
[ERROR]   Run 5: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [sample.tap]
[INFO]   Run 6: PASS
[INFO]   Run 7: PASS
[ERROR]   Run 8: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [4.tap]
[ERROR]   Run 9: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [plan_comment_tr_footer.tap]
[INFO]   Run 10: PASS
[INFO]   Run 11: PASS
[INFO]   Run 12: PASS
[INFO]   Run 13: PASS
[INFO]   Run 14: PASS
[INFO]   Run 15: PASS
[ERROR]   Run 16: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [org.tap4j.testng.konobi.tap]
[ERROR]   Run 17: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [tap_with_diagnostic_and_without_lastparsedtestresult.tap]
[INFO]   Run 18: PASS
[ERROR]   Run 19: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [3.tap]
[ERROR]   Run 20: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [subtest.tap]
[ERROR]   Run 21: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [header_plan_tr_footer.tap]
[ERROR]   Run 22: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [tap_with_yaml_comments_bailout_directives.tap]
[INFO]   Run 23: PASS
[ERROR]   Run 24: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [tap_with_diagnostic_and_wrong_indentation.tap]
[INFO]   Run 25: PASS
[INFO]   Run 26: PASS
[INFO]   Run 27: PASS
[INFO]   Run 28: PASS
[INFO]   Run 29: PASS
[INFO]   Run 30: PASS
[ERROR]   Run 31: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-14-tap-stream.tap]
[ERROR]   Run 32: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [test-report.tap]
[ERROR]   Run 33: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-33_tap_stream.tap]
[ERROR]   Run 34: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-22-tap-stream.tap]
[ERROR]   Run 35: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-17-tap-stream.tap]
[ERROR]   Run 36: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [jsdom_test_result.tap]
[ERROR]   Run 37: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [phantomjs.tap]
[ERROR]   Run 38: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-15-tap-stream.tap]
[ERROR]   Run 39: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-12-tap-stream.tap]
[ERROR]   Run 40: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [npm-test.tap]
[ERROR]   Run 41: AllFilesTest.lambda$dynamicTests$2:78->parse:95 Runtime Errors parsing TAP file [issue-20-tap-stream.tap]
[INFO]   Run 42: PASS
[INFO]   Run 43: PASS

kinow avatar Dec 06 '22 18:12 kinow 

[ERROR] Errors: 
[ERROR] com.tupilabs.tap4j.AllFilesTest.dynamicTests
[INFO]   Run 1: PASS
[INFO]   Run 2: PASS
[INFO]   Run 3: PASS
[INFO]   Run 4: PASS
[INFO]   Run 5: PASS
[INFO]   Run 6: PASS
[INFO]   Run 7: PASS
[ERROR]   Run 8: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [4.tap]
[ERROR]   Run 9: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [plan_comment_tr_footer.tap]
[INFO]   Run 10: PASS
[INFO]   Run 11: PASS
[INFO]   Run 12: PASS
[INFO]   Run 13: PASS
[INFO]   Run 14: PASS
[INFO]   Run 15: PASS
[ERROR]   Run 16: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [org.tap4j.testng.konobi.tap]
[ERROR]   Run 17: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [tap_with_diagnostic_and_without_lastparsedtestresult.tap]
[INFO]   Run 18: PASS
[ERROR]   Run 19: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [3.tap]
[ERROR]   Run 20: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [subtest.tap]
[ERROR]   Run 21: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [header_plan_tr_footer.tap]
[ERROR]   Run 22: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [tap_with_yaml_comments_bailout_directives.tap]
[INFO]   Run 23: PASS
[ERROR]   Run 24: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [tap_with_diagnostic_and_wrong_indentation.tap]
[INFO]   Run 25: PASS
[INFO]   Run 26: PASS
[INFO]   Run 27: PASS
[INFO]   Run 28: PASS
[INFO]   Run 29: PASS
[INFO]   Run 30: PASS
[ERROR]   Run 31: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [issue-14-tap-stream.tap]
[ERROR]   Run 32: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [test-report.tap]
[ERROR]   Run 33: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [issue-33_tap_stream.tap]
[ERROR]   Run 34: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [issue-22-tap-stream.tap]
[ERROR]   Run 35: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [issue-17-tap-stream.tap]
[ERROR]   Run 36: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [jsdom_test_result.tap]
[ERROR]   Run 37: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [phantomjs.tap]
[ERROR]   Run 38: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [issue-15-tap-stream.tap]
[INFO]   Run 39: PASS
[ERROR]   Run 40: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [npm-test.tap]
[ERROR]   Run 41: AllFilesTest.lambda$dynamicTests$2:79->parse:96 Runtime Errors parsing TAP file [issue-20-tap-stream.tap]
[INFO]   Run 42: PASS
[INFO]   Run 43: PASS

kinow avatar Dec 06 '22 23:12 kinow 

[ERROR] Errors: 
[ERROR] com.tupilabs.tap4j.AllFilesTest.dynamicTests
[INFO]   Run 1: PASS
[INFO]   Run 2: PASS
[INFO]   Run 3: PASS
[INFO]   Run 4: PASS
[INFO]   Run 5: PASS
[INFO]   Run 6: PASS
[INFO]   Run 7: PASS
[ERROR]   Run 8: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [4.tap]
[ERROR]   Run 9: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [plan_comment_tr_footer.tap]
[INFO]   Run 10: PASS
[INFO]   Run 11: PASS
[INFO]   Run 12: PASS
[INFO]   Run 13: PASS
[INFO]   Run 14: PASS
[INFO]   Run 15: PASS
[ERROR]   Run 16: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [tap_with_diagnostic_and_without_lastparsedtestresult.tap]
[INFO]   Run 17: PASS
[ERROR]   Run 18: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [3.tap]
[ERROR]   Run 19: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [subtest.tap]
[ERROR]   Run 20: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [header_plan_tr_footer.tap]
[ERROR]   Run 21: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [tap_with_yaml_comments_bailout_directives.tap]
[INFO]   Run 22: PASS
[ERROR]   Run 23: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [tap_with_diagnostic_and_wrong_indentation.tap]
[INFO]   Run 24: PASS
[INFO]   Run 25: PASS
[INFO]   Run 26: PASS
[INFO]   Run 27: PASS
[INFO]   Run 28: PASS
[INFO]   Run 29: PASS
[ERROR]   Run 30: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [issue-14-tap-stream.tap]
[ERROR]   Run 31: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [test-report.tap]
[ERROR]   Run 32: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [issue-33_tap_stream.tap]
[ERROR]   Run 33: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [issue-22-tap-stream.tap]
[ERROR]   Run 34: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [issue-17-tap-stream.tap]
[INFO]   Run 35: PASS
[ERROR]   Run 36: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [issue-15-tap-stream.tap]
[INFO]   Run 37: PASS
[ERROR]   Run 38: AllFilesTest.lambda$dynamicTests$2:85->parse:102 Runtime Errors parsing TAP file [issue-20-tap-stream.tap]
[INFO]   Run 39: PASS
[INFO]   Run 40: PASS
[INFO] 
[INFO] 
[ERROR] Tests run: 167, Failures: 0, Errors: 1, Skipped: 0

kinow avatar Dec 06 '22 23:12 kinow

Not sure if parboiled will be maintained in the future, nor if it works with Java 21... maybe consider https://github.com/zhong-j-yu/rekex or a hand-written recursive parser.

kinow avatar Nov 04 '23 11:11 kinow

Note to self: There's one example of PEG + Prett, not sure if that will help here (can't recall where I stopped), but in case it helps: https://news.ycombinator.com/item?id=39066465

kinow avatar Jan 21 '24 11:01 kinow