Why a syntax extension?

[dfreeman]

Apologies if the answer to this is of the form "well obviously..." -- I don't have a ton of experience working with Rust yet. I'm interested in understanding why you implemented this as a syntax extension instead of just using macro_rules.

For comparison, I've got a little helper I use for testing in a project that (I think) yields pretty similar results:

use serialize::json::{mod, ToJson};

macro_rules! json {
  (null) => (json::Null);
  ([ $($values:tt),* ]) => (json::List(vec![ $(json!($values)),* ]));
  ({ $($keys:expr : $values:tt),* }) => ({
    let kv_pairs = vec![ $(($keys.to_string(), json!($values))),* ];
    json::Object(kv_pairs.into_iter().collect())
  });
  ($ex:expr) => ($ex.to_json());
}

A couple interesting bits:

• Negative numbers are a problem here, too (and the parens workaround works)
• Every numeric literal needs a suffix, e.g. json!({ "list": [1i, 2, 3] }) won't work. It has to be one of:
  • json!({ "list": [1i, 2i, 3i] })
  • json!({ "list": (vec![1i, 2, 3]) })
• String slice literals don't work out of the box (since they don't have a ToJson impl), but it's easy enough to create a wrapper trait with impls for T: ToJson and &str.

Are you planning on extending the syntax (e.g. for expression splicing) in ways that would be more difficult to handle with macro_rules?

[tomjakubowski]

Fair question! I started on this several months ago and left it alone for a while so I can't remember why I decided to use a compiler plugin, but I do remember trying to use a "macro by example" and hitting a wall. There might've been a bug in macro_rules! or I might've not been clever enough at using them :smile:

I do know that the plugin offers some advantages:

• It's nice to be able to elide numeric literal suffixes, especially if you're pasting some blob of JSON in from somewhere for a test case or something. Negative numeric literals are still a wart, but that should be a relatively easy fix.
• A plugin has total control over parsing the token tree, which means we can theoretically provide better error messages than a MBE. For an example of how MBE error messages can be confusing, see the message given for json!({ "foo": "bar", }), where the problem is the trailing comma.
• I have some ideas for other JSON macros that would work well in this package which might not be possible as MBEs.

[tomjakubowski]

(by the way, a ToJson impl for str is pending: rust-lang/rust#18949)

[dfreeman]

Makes perfect sense -- thanks for taking the time to explain! (And for the str PR :smile:)

[tomjakubowski]

One huge advantage a MBE has is that they'll be stabilized in some form before 1.0 and therefore be available in Rust's stable releases. Compiler plugins may not be stabilized for some time. I think a less ergonomic, but usable in stable Rust, version of json! would make for a very nice library.

[tomjakubowski]

I'm re-opening this because I'd like to figure out if we can ship both a compiler plugin and a macro in the same library, with Cargo features or something like it determining which is built (on the stable channel, the macro would be used until a compiler plugin interface is stabilized).

[plietar]

I've added syntex support onto my fork : https://github.com/plietar/json_macros It's not as easy to use as a plain macro, but it lets it be used on Rust stable.

[dtolnay]

I implemented a stable macro_rules json! macro in serde_json 0.9:

https://docs.serde.rs/serde_json/macro.json.html

Some neat things about it:

• macro_rules, works great on stable Rust
• no parens needed around expressions
• negative numbers work
• allows interpolation of map keys, not just values
• allows optional trailing commas

I filed #34 to consider dropping Serde support in json_macros and focusing on rustc-serialize only.