Symbol encoding

[snunez1]

I am writing a plist DSL for possibly non-technical users that is translated via yason to a JSON specification. Simple example:

'(:title "foo")
 (:command my-command)
...)

where my-command is a symbol whose property list I need to encode as part of the transformation.

My first thought was to subclass symbol and then write a specialised encode method for it, but this implementation (SBCL) doesn't allow base classes as super classes.

The documentation for *symbol-encoder*:

symbol-encoder Function to call to translate a CL symbol into a JSON string. The default is to error out, to provide backwards-compatible behaviour.

A useful function that can be bound to this variable is YASON:ENCODE-SYMBOL-AS-LOWERCASE.

seems to suggest that I could write a streaming method and then bind it to *symbol-encoder*, however in the case of yason:encode ((object symbol)) ..., *symbol-encoder* simply returns a string for further processing and not JSON (and there's no available stream either).

What I've managed to get working is to bind to *symbol-encoder* a function that extracts the properties and writes them out as a plist, or string for symbols that don't have interesting properties for this application. However, this requires me to modify yason's encoder for symbols to relax the assertion for a string value. It doesn't seem like there's an easy way for me to override the processing for symbols.

I'd rather not ask users to download a custom version of yason, and would like to know if I missed other options, or if there's a more idiomatic way to do this. If there's no better way to do this, could we make the assertion accept a/p-lists as well as strings? (or perhaps anything that could be further encoded by yason)

[phmarek]

Hi,

I'd like to keep an ASSERT there to (kind-of) ensure a sane structure of JSON output. But I pushed an escape mechanism (not yet exported, though); how about something like this?

(defun enc (sy)
  (if (keywordp sy)
      (symbol-name sy)
      (yason::make-raw-json-output
        (yason:with-output-to-string* ()
          (yason:encode-plist (list* :name (symbol-name sy) (symbol-plist sy)))))))

(setf (get 'enc :foo) 51)

(let ((yason:*symbol-key-encoder* #'yason:encode-symbol-as-lowercase)
      (yason:*symbol-encoder* #'enc))
  (yason:with-output-to-string* ()
    (yason:encode (vector 1 "two" :cl 'enc))))

[snunez1]

That gets us part-way there, however the symbol-plist requires processing. In most cases it's not going to be ready to send to yason without some massaging.

[phmarek]

Well, you can do that in ENC, instead of just using SYMBOL-PLIST directly, right?

Or what is still missing?

[snunez1]

So, in this example, the function bound to yason:*symbol-encoder* should return a string that is either the symbol name or a string containing a JSON encoding of the symbol plist?

[phmarek]

Either a string that gets stored as string in JSON, or some other data via YASON::MAKE-RAW-JSON-OUTPUT that gets inserted verbatim.

See my ENC function.

[snunez1]

So far, so good.

Is the intention to export make-raw-json-output ?

[snunez1]

I've encountered a case that's isn't catered for: encoding symbols to JSON null. Since the output must be a string, the best I can get is "null", but that's not the same.

Is there a way to output things like nil and :na as null?

[phmarek]

Well, I guess you'd have to write your own function to use as *symbol-encoder*, to accomodate your (local) null symbols...

YASON currently only uses :null.

[phmarek]

Addendum: you'll need to use something like (encode (make-raw-json-output "null")).