Comments

[Bike]

Sup. This is long, sorry. First I'd like to mention my appreciation that your ideas here are more uh... how do I put this... usually when I've seen ideas for standard extension they haven't been very coherent, and seem to be more interested in following the pack of other languages than seriously considering what problems Lisp actually has. So, nice.

Secondly, I'm going to mention previous/alternate versions of several things you mention in the README, that don't seem to be mentioned. I don't know if you know of them or not, but I think they are useful cross references.

Anyway:

• LOOP: you probably know about [iterate](https://common-lisp.net/project/iterate/ which is extensible). You may or may not know that the Xerox implementation of LOOP used in several implementations is extensible. This extensibility is not really documented but it is there, and has been used for example to implement SBCL's for-as-sequence clause. Looking at how that's done would probably be good for any specification of an extensible LOOP.
• Allow eval access to environment. This is an idea I very much support in principle but it would require some major changes. Environment objects in the standard are basically "compile-time", and do not really include things like the actual values of variable bindings. Actual runtime environments would be their own beasts. The most interesting system with really usable runtime environments is, I think, Kernel and I have a few partially baked ideas relating to a language with real runtime environment introspection and usage.
• Compilation/block compilation. I have some even less baked ideas here. I think that COMPILE-FILE and COMPILE could be provided with a (compile-time kind) environment. Then, the semantic constraints in CLHS 3.2.2.3 would apply only with respect to that environment. This would allow precise control of what a compilation unit is actually dependent on. Many additional operators would be required, for example to construct one of these environments from a file (sort of like a runtime CFASL) and read and alter them finely.
• Extensible sequences. I assume you are already aware of the existing sequences extension which is implemented by SBCL, Clasp, and ABCL. I think that it is pretty good though I have some minor issues with it, which I have commented on a bit on Clasp's implementation. I have also written a bit of a more general extension for "iterators", which allow the use of some sequence function on things that are not sequences (e.g. because they are infinite, like lazy lists, or unordered, like sets or hash tables). Iterators
• Standardize the Meta-Object Protocol for CLOS. Something else I support in principle, but there is a major issue: The meta-object protocol as described in AMOP is not adequately specified and has some weird holes. I mean if you look at Closer-MOP you can see it has pretty extensive code - it's not just a portability library, it's trying to hack pretty different implementations into something uniform.
• First-class macros. This is another thing I have been interested in and done some work on. In ancient lisp dialects they were called fexprs. They are interesting but there are some serious issues with them. First, fexpr forms cannot be compiled without specific per-fexpr support from the compiler. Second, if the implementation is not permitted to assume that unknown operators are functions (rather than macros or fexprs), it cannot compile much of anything. Third, they cause some weird syntax escaping, e.g. if in a scheme with fexprs you write (define (funcall1 op arg) (op arg)), (funcall1 a-fexpr x) will result in the fexpr receiving the symbol arg, from the definition of funcall, as an argument. Fourth, I haven't seen many uses for them that wouldn't be covered just by using a macro. Kernel, mentioned above, is the most extensive modern system with fexprs I know of.
• Environments. See Trucler
• Name conflicts. This is already implemented widely enough that it could probably be a CDR or whatnot.
• Object system
  • I have some ideas about a replacement for defstruct that I'll elaborate on in my repo. The short version is that I would like to have structs that can be stored unboxed in arrays, or ideally, usable to implement your own arrays efficiently, with something like C's flexible array members.
  • Recursive types seem nice but involve subtle issues with a dynamic typing system with subtypes. One I have contemplated often: the proper list type is defined by proper-list = (or null (cons t proper-list)). We could also define a type for circular lists: circular-list = (cons t circular-list). This seems reasonably intuitive, but what is (subtypep 'circular-list nil)? With the (sorta) obvious definition of subtypep on recursive types, we would go, okay, let's assume it is, and then it's a subtype if (cons t circular-list) is a subtype of nil; this is a subtype if the cdr of the cons is a subtype of nil; we are asking whether (subtypep 'circular-list nil) is true while determining whether (subtypep 'circular-list nil) is, so by assumption it is, and so circular lists don't exist. Sort of the issue here is that familiar recursive types in Haskell and so on are isorecursive, while in Lisp equirecursive types seem more obvious. https://en.wikipedia.org/wiki/Recursive_data_type#Theory
  • The problem with specializing on types is that there is no obvious ordering of them. The ordering of classes is actually a very important but subtle part of CLOS: Every finalized class has a class precedence list which controls which methods are selected. You can see the problem here if you imagine having one method specialized on (integer 0 1) and another on (integer 1 2); which is called first? This is before even getting into the efficiency concerns.
  • Robert Strandh (beach) has developed a new system for performing generic function dispatch very efficiently. To summarize, the discriminating function is JIT-compiled. Once a generic function has been called a few times, the discriminating part is basically just a jump table. It's very fast.
  • I have some ideas on top of this to completely eliminate discrimination in some contexts; for example if a method is specialized on a class, and the method calls some accessor, the accessor could just be inlined. I don't think anything in the standard prevents this, but careful work needs to be done to ensure code isn't bloated, and it would introduce extensive interconnection between generic functions in order to ensure everything is updated when it needs to be.
  • Christophe Rhodes, the author of the sequences extension, also had some ideas about allowing MOP users to decide on their own how their generic functions should discriminate: Generalizers
  • Traits/typeclasses - I really don't see how these make sense outside of a statically typed system - and not Lisp's gradual typing system, which is inherently optional.
• Syntax. I'm not sure I understand some of what you're getting at here, but I will mention that the standard already specifically defines which reader macro characters can be introduced by programmers. CLHS 2.1.4 explicitly reserves (for example) [] and {} for user use, while ~ is reserved to implementors. I will also mention that the reader implementation Eclector includes more hooks for the reader that could be incorporated into an extension, e.g. here.
• Separation into libraries. This would be very difficult, because there are many cross-cutting concerns. For example practically every standard function can signal errors in some situation, meaning making the condition system optional is difficult.
• Continuations. I'm fond of delimited continuations, which I think are better than undelimited ones. Some interesting resources here are this fella's work and the very complex continuation system in Racket, which includes some stuff that is probably intended to allay concerns like Pitman's.

[commander-trashdin]

I think I'm just currently addressing the part about extensible sequences (in a sense) as well as part about dispatching on types and typeclasses in my https://github.com/lisp-polymorph(btw we still need any help we can get with it) -- which is why I stopped adding stuff to this doc. Doesn't mean I abandoned it btw. Thank you for your input, if you think you can extract something and add it to the doc itself, I would appreciate it a lot.