convex
convex copied to clipboard
Convex-Dev
Mutability with `set!` is counterintuitive with closures
Coming from Clojure, altering a let binding looked kind of wrong but it is certainly beginner-friendly for people unfamiliar with immutability. Furthermore, Clojure does have some mechanisms for cheap mutability (ie. volatiles).
The problem is that this sort of side-effects should outlive the current scope and work with closures. At least, I assume that this is expected behavior since this is how mutating a variable works in languages supporting closures.
Suppose this function which is supposed to increment x each time:
(def f
(let [x 0]
(fn []
(set! x (inc x))
x)))
Calling f repeatedly always returns 1 instead of incrementing. The changes applied to bindings in f are lost. If you read carefully the docstring of set! then this is perfectly valid. But isn't it error-prone?
The broader question: is it really worth mutating local bindings?
Interesting. I think it depends on the semantic definition of a local binding, which is currently a lexically defined binding on the call stack. We could potentially drop set! though it is convenient and efficient in some circumstances (where otherwise you would need to do a bunch of conditionals and/or a loop/recur construction. Needs some more design thought.
If it is about looping, then I would enforce using recur with the bindings that get mutated instead of (recur) and using set!. It is not only more "functional", it helps in spotting right away what might change or not (all changes are unambiguously located in recur).
Without looping, do you have a simple example of what would be problematic or less comfortable? I might be too used to Clojure and have developed a Stockholm syndrome. You can actually overwrite a local binding in let if you want to. In practice I never do, I always write x, x-2, x-3, ... So that I can see right away which changes apply where.
Not just about looping. My canonical use case is for things that conditionally update a binding e.g..
(fn [x y]
(if (some-condition? x y) (set! y (foo x y)))
(if (some-other-condition? x y) (set! x (bar x y)))
(if (yet-another-condition? x y) (set! y (baz x y)))
(do-the-main-event x y))
Yes, you can do with let bindings I guess but it is a bit ugly and means you must always update the binding, even if the condition is rare and you usually update it to be the same value.
Okay, this is one example where mutability is more efficient and there is no way to keep it immutable without sacrificing that (I don't think so).
But that kind of code is fairly rare in my experience. With more context, there might be better ways for handling that kind of scenario without resorting to this. And you could argue that rebinding, or binding to another symbol (eg. x -> x-2) is less efficient but more readable. Which is important since Convex is more concerned with human readability than extreme number crunching.
Regardless, there is a case for having local mutability and not having to resort to def. Even something like a transducer-like API (mentioned in #117) would need a stateful mechanism.
An alternative solution would be to have an explicit stateful boxing type (akin to atom, or rather volatile). What I like about a box is that mutability is confined. What you can mutate or not is always clear, as opposed to set! allowing anything to become anything, adding extra mental load. Mutability remains easy, but not so easy that users would prefer it over immutable constructs for looping etc.
Plus, I guess that a box would be a friendlier approach for closures. Easier to implement, as opposed to re-thinking what local bindings are, how to mutate them from a closure, ...
Another proof that something is not quite right about set!:
(let [x 0] (dotimes [i 10] (set! x (inc x))) x)
It is safe to assume that anyone would assume the result to be 10 whereas it is 0 in reality. I guess this is because dotimes desugars to loop which creates another point of local bindings.
Similarly to the following returning 10 instead of 42:
(let [x 10] (let [] (set! x 42)) x)
So, implementing a mutable box type looks like the simplest solution.
So... we can't really support mutable boxes in Convex (at least as first class values) since these would violate the core assumption that everything which can be stored in an environment is an immutable value.
Perhaps we could limit set! to only modify a binding that is defined in the innermost binding form. Would require some extra compiler support to track this, though we probably need to do this anyway to support types and fully hygenic macros later....
I think about it once in a while, talked about it with @pbaille... I would suggest either having real mutability (but we can't, so...) or no mutability at all. There might be a couple of use cases like you provided but I wouldn't say it's such a big deal. On the other hand, it has (very?) unexpected behavior for people accustomed to mutable closures and trying to make it marginally safer would add more complexity to the compiler.
So this one might change if we implement local tracking in the compiler (currently we don't). If this was the case, we could enforce things like "set! only works on a defined local binding in the current enclosing binding form".
Alternatively, we could go for a full locals stack, and let set! locally modify any lexically bound variable. This is hairy but doable I think.
Actually mutating something in a Closure is no-go (they need to be immutable). This is really about changing a binding in the local execution context.
Yes, maybe I should have said that from the beginning but I was meaning "mutable" like def is kind of mutable. I really think that any other way is suboptimal and error-prone.
I think this one for documenting set! very clearly say that set! only affects the current local binding