Build a test suite

[kavitharaju]

For the implementation and testing of the python parser, as well as the support for different USFM features by the underlying grammar, we need to build a large test suite covering at least all test cases we had in V2.

This should ideally include, input USFMs, expected outputs in different formats like JSONs, CSVs, USX etc

[kavitharaju]

Using USFM/X committee's test suite.

Current status https://github.com/Bridgeconn/usfm-grammar/pull/182#issuecomment-1283951776

[kavitharaju]

Current Status

• Total USFM samples in the test suite = 249
  • No. of positive tests = 197/249
  • Correctly parsing(Generating syntax tree and/or errors) = 184/249
    • The reason for failure of 65 samples are recorded here
    • No of positive tests among these are 154
  • Able to generate JSON without raising error for 154/154 (Correctness of JSON is not yet verified)
  • Able to generate USX without raising error for 154/154
  • Generated USX confirms to the USX.rnc grammar(With a few modifications) for 86/154
    • The reason for failure of 68 cases are recorded here
  • Comparing with the USX sample in test suite
    • No. of USX files in test suite passing validatio by the USX.rnc grammar = 81/154
    • No. of cases where our generated USX == Origin.xml of the testsuite = 21/154

[kavitharaju]

Another update https://github.com/usfm-bible/tcdocs/pull/44

[kavitharaju]

Upon pulling latest tests from tcdocs, finding these grammar rule changes in test suite even though https://docs.usfm.bible/usfm-usx-docs/latest/index.html doesn't reflect most of them yet:

1. \fig is allowed in \f, as per this testcase, which is stated as a newly supported feature in 3.0 and 3.1.
2. gloss is a required attribute in \rb as per this testcase. But this testcase contradicts it.
3. who is the default attribute of \qt-s, as per this testcase.
4. Some unidentified issue with \imt# or \imte#, in this testcase.
5. \cat is now allowed witthin \f, immediately after caller, as per this testcase.
6. Empty \w \w*, is supposed to fail now, as per this testcase.
7. Empty book with only \id line are supposed to fail now, as per this testcase. Does it make any specific type of makers mandatory like \p or does it simply require something else along with \id? Will one of header markers, intro markers, comments(\rem) or \c do?
8. Newly added \ref marker, as seen in this testcase.
   • :warning: But its USX is different from other markers. with type=ref instead of char and no style attribute <ref loc="MRK 1:4">Mark 1:4</ref>
9. Allow both \xt and \+xt within footnotes. As per this tescase and this testcase.
10. Allow attributes without link- prefix (href, id etc.)