nfiles icon indicating copy to clipboard operation
nfiles copied to clipboard

Searialize lisp-file contents to `read`-able data?

Open aartaka opened this issue 2 years ago • 7 comments

How about this: define a reader macro for classes and (de)serialize objects with this macro in *readtable*? Something like sharpsign-s reader macro for structures, but for classes. Maybe even rebind the structure one? Then we can rebind readtables to the new syntax for the time or reading:

(defmethod files:serialize ((profile profile) (file lisp-file) stream &key)
  (let ((*readtable* (copy-readtable)))
    (set-dispatch-macro-character #\# #\S #'read-class-or-structure)
    ;; A hypothetical function writing the object with the #S syntax.
    (write-nested-maybe-objects (content file) :stream stream)))

(defmethod files:deserialize ((profile profile) (file lisp-file) stream &key)
  (declare (ignore stream))
  (let ((*readtable* (copy-readtable)))
    (set-dispatch-macro-character #\# #\S #'read-class-or-structure)
    ;; A hypothetical function writing the object with the #S syntax.
    (read (call-next-method))))

Benefits:

  • Concise (much more than cl-prevalence, I'm guessing).
  • Optimized as much as read is (a lot, I'm guessing).
  • Looks familliar.
  • No dependencies.

Downsides:

  • Non-portable.
  • Requires a custom printer (or can we somehow instrementalize Lisp printer to output the new syntax? I wouldn't be surprized if we actually can...)

aartaka avatar Sep 06 '22 14:09 aartaka

If I get it right, you want to skip the deserialize call so that read-ing would be enough, is this correct?

In any case, we would not replace cl-prevalence, since write-nested-maybe-objects is essentially what cl-prevalence does.

Note: can we leverage make-load-form?

Ambrevar avatar Sep 07 '22 11:09 Ambrevar

Looks interesting. You never know how much foresight you can discover in the spec ( ͡° ͜ʖ ͡°)

EDIT: an -> can

aartaka avatar Sep 07 '22 13:09 aartaka

Actually make-load-form would only work if we load the file instead of merely read-ing it. Not sure that would fit the bill here...

Ambrevar avatar Sep 07 '22 13:09 Ambrevar

Actually make-load-form would only work if we load the file instead of merely read-ing it. Not sure that would fit the bill here...

That's would work perfectly in conjunction with #5! Imagine: we serialize the data to a huge make-load-form output, write it to a file, complile to fasl, and then load it, getting the whole set of objects restored! \(@ ̄∇ ̄@)/

aartaka avatar Sep 07 '22 14:09 aartaka

A small demo that seems to work on simple/standard types:

(defvar *data* nil)

(defun object->file (object file)
  (with-open-file (f file :direction :output
                          :if-does-not-exist :create
                          :if-exists :supersede)
    (prog1
        (write `(setf *data* ,(make-load-form object)) :stream f)
      ;; Optional, to speed things up.
      (compile-file file)
      file)))

(defun file->object (file)
  (load file)
  *data*)

aartaka avatar Sep 07 '22 15:09 aartaka

OK, but what abou the readtable trick from the original post?

Ambrevar avatar Sep 12 '22 09:09 Ambrevar

Also before working on this, let's run some benchmarks (e.g. on Nyxt's GHT).

Ambrevar avatar Sep 12 '22 09:09 Ambrevar