Lerche.jl
Lerche.jl copied to clipboard
Allow immediate transformation during parsing
Building parse trees from very large files (e.g.> 200MB data file) can occupy considerable memory. If the transform is applied immediately at the point that the syntactical construct is recognised, memory can be economised, particularly if only certain information is relevant. This would also allow streaming data. Lark already has this capability via the transform
option to Lark
, which in Lerche
is simply executed following the completion of parsing.
Considerations:
- If an item for the parse tree is processed by user-supplied functions, the type restraints for nodes of
{Tree,Token}
are no longer true. Some form of type parameterisation is needed to avoid type instability. This should be under the control of the user.