ivy icon indicating copy to clipboard operation
ivy copied to clipboard

ivy: operator changes break stored parses

Open rsc opened this issue 1 year ago • 0 comments

Consider this transcript:

op g x = f x
f=1
g 2
	1 2

op f x = 1+x
cannot define op f; it is a variable (f=0 to clear)

f=0
op f x = 1+x

g 2
	undefined global variable "f"

op g x = f x
g 2
	3

This is awkward because when g x is parsed, Ivy "decides" that f is a reference to a global. It must make that decision in order to store a full parse of g. But it need not store a full parse of g. It could instead treat any sequence of tokens as a List expression and evaluate the exact meaning of that list at Eval time. So 'op g x = f x' would store something like List(Name(f), Name(x)) as the parse, and then List would Eval by evaluating each thing in the list, right-to-left, putting together the meaning as it went.

I've been thinking about how to handle things like higher-level functions (for example a user-defined 'rep' such that (f rep 3) x means f f f x). With the current AST, the parser needs to know whether (f rep 3) is a function (making (f rep 3) x a call) or data (making (f rep 3) x a vector). If instead the parser just treats it as a list of unspecified meaning, it can wait until eval time to figure out the exact meaning: run 'f rep 3' and see what kind of thing it returns.

Not suggesting any particular changes now, but I wanted to write down the insight that more functional features might require less aggressive parsing.

rsc avatar Jan 14 '25 05:01 rsc