Add a simple package manager for lisp/lem libraries

[Sasanidas]

This is the initial proposal of a simple package manager, this implementations contains the following features:

• A simple and declarative way of defining an external package (in this context, "package" is basically an asd system), both using quicklisp and git (with the possibility of using a custom branch and/or commit) as a source:

https://github.com/lem-project/lem/assets/26670956/51c503e0-6878-44a5-957d-72756b2244bf

• Two commands to manage packages (install/remove) from quicklisp and from git:

https://github.com/lem-project/lem/assets/26670956/ddbcda58-eb38-4323-afa5-56d3eab16b8e

I chose to use a folder in "~/.lem/packages" to save the packages, the dependencies of the packages will be managed by quicklisp. This approach helps developing external packages while using Lem (I can load packages from git, modify them and push the changes to the package repository).

I'm planning to extend the lem-use-package options in the future to add more useful features.

(The idea of this simple approach is to just have something to manage packages until https://github.com/lem-project/lem-extension-manager is finished)

[vindarel]

and my feedback:

• use a shorter package name? (or more fun :p )
• in run-git or the download functions: no need of more error handling? (network error, the repository doesn't exist etc)
• eval-when: will this be run at compile time? It should be run when the user launches the package manager.
• I don't undesrtand the names spackage, rsource, pdir, dfsource, rpackage.
• what about splitting the macro: extract the flet, use a call-with- pattern?
• write an accompanying user-level documentation, that we would keep in sync with the source and copy-paste to lem-pages?

[Sasanidas]

• use a shorter package name? (or more fun :p )

Ummm, I think simple-package is good enough, I'm not that creative when it comes to names :sweat_smile:

• in run-git or the download functions: no need of more error handling? (network error, the repository doesn't exist etc)

I wanted the error to just rise to the interface (which Lem can handle and show to the user), maybe I can catch some and improve the messages, but I think they are quite clear most of them.

• eval-when: will this be run at compile time? It should be run when the user launches the package manager.

Indeed I don't think it's necessary the eval-when, but this should not be updated every time the user launches a command, it's most like a cache for ql packages for auto-completion.

• I don't undesrtand the names spackage, rsource, pdir, dfsource, rpackage.

spackage = simple-package rsource = it's the local source defined pdir = package directory dfsource = define-source rpackage = it's the local name for the package

They are kind of inferred from the context, but indeed this style of naming is not usual.

• what about splitting the macro: extract the flet, use a call-with- pattern?

Indeed, I'll move the flet to a separate, about the call-with-pattern, I'll add it for the configuration of the downloaded package (basically the s-expression of the configuration keyword that is not yet available).

• write an accompanying user-level documentation, that we would keep in sync with the source and copy-paste to lem-pages?

Indeed! I'm planning to document everything, I just wanted the feedback of Sasaki-san first maybe he wish to make some changes or have other requests.

[vindarel]

eval-when […] it's most like a cache for ql packages for auto-completion.

then a simple variable not instantiated, to check with boundp? (to differentiate with NIL) This seems nicer and simpler… and more correct? :thinking:

[Sasanidas]

then a simple variable not instantiated, to check with boundp? (to differentiate with NIL) This seems nicer and simpler… and more correct? 🤔

I'm using it as a constant basically (not really a constant, but is not meant to change), it should get populated when compiling, which given the rate of update of quicklisp should be enough I think. I'll add a command to update the packages list in the future.

[vindarel]

I'm using it as a constant basically (not really a constant, but is not meant to change), it should get populated when compiling, which given the rate of update of quicklisp should be enough I think. I'll add a command to update the packages list in the future.

so there can be a difference between the compile-time list and the user's Quicklisp dist. It's maybe less an issue than I think, specially with a command to update it.

no more questions and remarks, thanks. Great addition.

[Sasanidas]

There is an interesting problem that I would have to address. When calling the lem-use-package, is there code that uses the packages that it loads, it will trigger a read-time error (as at read time the package itself hasn't been loaded yet).

So, my idea is to do a macro-reader to load the package at read-time , so any reference below can work without problems

[Sasanidas]

So, to not add more complexity, I just added a function to load the initial files (instead of doing it using a custom asd system). This will allow to load the file at read time with the built-in read macro #., so lem-use-package have to be use in a loaded file:

#.(lem/simple-package:lem-use-package
   "lisp-critic"
   :source '(:type :git
             :url "https://github.com/g000001/lisp-critic.git"))

(define-command fer/critic-reference () ()
  (let ((beg nil)
        (end nil)
        (output-string nil))
    (lem:with-point ((p (current-point)))
      (funcall
       (variable-value 'lem/language-mode:beginning-of-defun-function :buffer)
       p 1)
      (setf beg (copy-point p)
            end (lem-vi-mode/commands::vi-forward-matching-paren
                 (current-window) p)
            output-string (with-output-to-string (s)
                            (let ((*standard-output* s))
                              (lisp-critic:critique-definition
                               (read-from-string
                                (format nil "~a)"(points-to-string beg end)))))))
      (fer/describe-thing
       (or (and (not (str:emptyp output-string)) output-string)
           "All is well!")))))

(in this example, the file has to be loaded, otherwise the reader may fail if it load the file from the cache)

[Sasanidas]

I changed the name of the commands to use the prefix sp as I think its more easy to use that way

[Sasanidas]

So I think the functionality of this PR is finish and ready for testing. I'll document everything in the webpage after merge-time :)

[Sasanidas]

I'll move this functionality to https://github.com/lem-project/lem-extension-manager