Check allowed characters in replace

[smedilol]

Describe the bug

Generatlization of #2471. IBM Compiler check characters allowed in replace clause. See "Chapter 1. Characters" page 3 of IBM PDF "Enterprise COBOL for z/OS 6.3 Language Reference".

Allowed characters are :

(space)
+
-
*
/
=
$ (currency sign). TODO check if it can be changed inside replace directive.
,
;
.
'
"
(
)
<
>
:
_
A-Z
a-z
0-9

So for example we must forbid character @ in replace like IBM compiler.

To Reproduce

CheckPseudoText2.rdz.cbl:

       IDENTIFICATION DIVISION.
       PROGRAM-ID.    CheckPseudoText2.

       ENVIRONMENT DIVISION.
       CONFIGURATION SECTION.
       SOURCE-COMPUTER. IBM-370.
      *SOURCE-COMPUTER. IBM-370 WITH DEBUGGING MODE.
       OBJECT-COMPUTER. IBM-370.
       SPECIAL-NAMES.
                        DECIMAL-POINT IS COMMA.

       DATA DIVISION.
       working-storage section.
       01 Var100 pic 9(05).

       01  Group1.
           05 filler pic X.


       replace ==:D:== by ==_==.
           05 A:D:Var pic X.


       replace ==:D:== by ==-==.
           05 A:D:Var pic X.


       replace ==:D:== by ==.==.
           05 Var pic X :D:

       replace ==:D:== by ==,==.
           05 Var pic 9(5)V9(2) value 1:D:5.


        replace ==:D:== by ==)==.
           05 Var pic X(2:D:.


       replace ==:D:== by ==(==.
           05 Var pic X:D:2).


       replace ==:D:== by ==  ==.
           05 :D:Var pic X.
           05:D: Var pic X.



       01 filler pic X.

       procedure division.
       replace ==:D:== by ==>==.
           if Var100 :D: 2
               display ">"
           end-if


       replace ==:D:== by =====.
           compute Var100 :D: 1 + 2


       replace ==:D:== by ==<==.
           if Var100 :D: 2
               display "<"
           end-if

       replace ==:D:== by ==/==.
           compute Var100 = 2 :D: 1


         replace ==:D:== by ==+==.
           compute Var100 = 2 :D: 1


       replace ==:D:== by ==*==.
           compute Var100 = 2 :D: 1


       replace ==:D:== by ==;==.
       replace ==:D:== by ==$==.

      *The"REPLACE"statement was invalid.
      *  Expected"==", but found":".  The statement was discarded.
       replace ==:D:== by =='==.

      *Note : IBM compiler still to find the end of the pseudo-text
      *delimiter ==.
      *If the following line is commented, then the rest of the text
      *will be considered as part of the pseudo-text 2 of the replace.
           ==.

      *We don't need to reproduce this behavior. It's just explained in
      *case you want to reproduce this test with IBM Compiler.


      *    Simple instruction to act as a catcher of potential misplaced
      *    errors of IBM compiler.
           display "---"
      *Ok
       replace ==:D:== by ==''==.


      *An invalid"REPLACE"statement was found.
      *  Scanning was resumed at the period terminating the
      *  "REPLACE"statement.
      *The pseudo-text ending delimiter"=="was missing in a
      *  "REPLACE"statement.  The"REPLACE"statement was discarded.
       replace ==:D:== by =="==.
      *same as previous replace, uncomment following line to test
      *with IBM Compiler
           ==.

      *Ok
       replace ==:D:== by ==""==.


      *Non-COBOL character"@"was found in column 29.
      * The character was accepted.
       replace ==:D:== by ==@==.

           goback
           .

       END PROGRAM CheckPseudoText2.

Expected behavior

Mandatory: check that characters inside a partial cobol word are valid.

Bonus : Same error as IBM compiler. IBM compiler sometimes seems to stop its parsing or try to find the end of the replace clause. We don't need to reproduce this behavior of course. It would be better to detect that the replace clause was intented to finish at a certain point and report that the replace is invalid.

Technical TODO Will be edited by a team member. Implementation details with the chosen solution.

How to test automatically Standard unit test with Token comparator.