add unimplemented nodes to AST instead of raising exceptions

[tomasohara]

To facilitate broader coverage of the analyzer, it would be good for the parser to add "unimplemented nodes" to the AST rather than raising an error. This can be done as follows:

$ git-diff bashlex/parser.py
...
+from mezcla import system
+
+ADD_UNIMPLEMENTED_NODE = system.getenv_bool("ADD_UNIMPLEMENTED_NODE", False,
+                                            "Add unimplemented nodes to parse tree")
+
 from bashlex import yacc, tokenizer, state, ast, subst, flags, errors, heredoc
 
 def _partsspan(parts):
@@ -13,14 +19,21 @@ precedence = (
 )
 
 def handleNotImplemented(p, type):
-    if len(p) == 2:
+    if ADD_UNIMPLEMENTED_NODE:
+        parts = _makeparts(p)
+        p[0] = ast.node(kind='unimplemented', parts=parts, pos=_partsspan(parts))
+    elif len(p) == 2:
         raise NotImplementedError('type = {%s}, token = {%s}' % (type, p[1]))
     else:
         raise NotImplementedError('type = {%s}, token = {%s}, parts = {%s}' % (type, p[1], p[2]))

This way, a parse tree can still be recovered even though a particular construct is not supported:

$ ADD_UNIMPLEMENTED_NODE=1 python -c 'import bashlex; print(bashlex.parse("case fu in esac")[0].dump())'
UnimplementedNode(pos=(0, 15), parts=[
  ReservedwordNode(pos=(0, 4), word='case'),
  WordNode(pos=(5, 7), word='fu'),
  ReservedwordNode(pos=(8, 10), word='in'),
  ReservedwordNode(pos=(11, 15), word='esac'),
])

I can add a pull request for this if you want.

[idank]

Sure sounds useful. Any reason you went with an environment variable rather than an argument to the parser?

[tomasohara]

Oh, that was just for the sake of a simple illustration. I'll add an option to the parse function instead.

Best, Tom

On Tue, May 30, 2023 at 1:32 AM Idan Kamara @.***> wrote:

Sure sounds useful. Any reason you went with an environment variable rather than an argument to the parser?

— Reply to this email directly, view it on GitHub https://github.com/idank/bashlex/issues/86#issuecomment-1567845499, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFCQ2GJR35HZYXSVYU2EZLXIWH6XANCNFSM6AAAAAAYTP37EA . You are receiving this because you authored the thread.Message ID: @.***>

[idank]

Thanks. Make sure you squash all the commits to one and I'll merge that in!