grammars-v4 icon indicating copy to clipboard operation
grammars-v4 copied to clipboard

[rexx] Parse hexstrings and binarystrings correctly per ANSI standard.

Open RossPatterson opened this issue 3 years ago • 1 comments

Parse hexstrings and binarystrings correctly per ANSI standard.

Fixes #2741.

  • Identify hexstrings and binarystrings in the parse tree by their own parser rules.
  • Do not accept hexstrings containing invalid hexadecimal characters.
  • Do not accept binarystrings containing invalid binary characters.
  • Accept blanks between pairs of hexadecimal digits, counting from the right (ANSI sec. 6.2.2.38 et seq.).
  • Accept blanks between 4-tuples of binary digits, counting from the right (ANSI sec. 6.2.2.41 et seq.).

In addition, replace the existing Java lexer action that identifies the trailing radix ("b", "B", "x", or "X") with pure-ANTLR lexer rules, to remove one of the only barriers to a target-agnostic grammar.

This change adds a new test_bitstring.rex set of test cases. There are "good" and "bad" cases in this set, obvious by their names (e.g., binary_good1 and hex_bad2). The "good" cases should parse to either binary_string or hex_string. The "bad" cases have different parsing interpretations, because they're invalid binary- or hex-strings, but they're valid as some other expression. There are also a few cases that are obviously not binary- or hex-strings (e.g., str_good3), and that should never parse as such (e.g., "hi mom!"x). mvn test will compare the parse of this file to a reference file (test_bitstring.rexx.tree), which has the expected parsing results.

Passes mvn test, including the new test cases.

RossPatterson avatar Aug 16 '22 02:08 RossPatterson

Apparently there are linend issues in the test-case-verification stuff. I'll sort that out and push an update, probably tomorrow.

RossPatterson avatar Aug 16 '22 02:08 RossPatterson

I'll reorganize and resubmit.

RossPatterson avatar Aug 17 '22 18:08 RossPatterson

Replaced by PR #2769.

RossPatterson avatar Aug 17 '22 18:08 RossPatterson