rune icon indicating copy to clipboard operation
rune copied to clipboard

Add support for underline `_` in number literals

Open SWW13 opened this issue 1 year ago • 5 comments

It would be nice to have underline _ support in number literals. This allows for easier reading of large numbers.

error: parse error
  ┌─ test.rn:1:20
  │
1 │ const ADDR = 0xDEAD_BEEF;
  │                    ^ expected `;`, but got `_`

Error: failed to build rune sources (see diagnostics for details)

SWW13 avatar Aug 12 '22 11:08 SWW13

This is implemented for integers in #432, but since float parsing uses a different routine (std::str::FromStr) we'd have to switch to a float parser which supports skipping over _.

This is where it would have to be changed if anyone finds the time to implement or find a library which works: https://github.com/rune-rs/rune/blob/db60f417d4b955a642efdadbb7b97f08d2dd3bfd/crates/rune/src/ast/lit_number.rs#L70

udoprog avatar Aug 29 '22 10:08 udoprog

Nice :)

For floats I see two possible ways to go:

  1. Add a "first-layer" parser before FromStr or e.g. fast-float which removes _.
  2. Built our own float parser and outsource the heavy part to minimal-lexical.

SWW13 avatar Aug 30 '22 09:08 SWW13

Any approach is fine with me. I prefer ones which avoid allocating if possible.

More data about the number can also be extracted during lexing if necessary beyond is_fractional, like text spans to extract components. Something it seems like minimal-lexical would be able to make use of.

udoprog avatar Aug 30 '22 10:08 udoprog

Also we haven't clearly defined which parsing standard we abide by. Lacking that it would be "whatever Rust does". I seem to recall there was some improvement posted about float parsing in the Rust compiler a month or two ago but can't find it. Maybe they are available as external crates?

udoprog avatar Aug 30 '22 10:08 udoprog

Allocation free should be easy with minimal-lexical, we can just wrap the number with a "skip _"-iter. This approach generally feels like a good fit.

Also we haven't clearly defined which parsing standard we abide by. Lacking that it would be "whatever Rust does".

This sounds like a sane default: "Where ever possible try to mimic Rust, only deviate if there are road-blocks or good reasons not to".

I seem to recall there was some improvement posted about float parsing in the Rust compiler a month or two ago but can't find it.

IIRC there was a post long ago (~1yr) about how the Rust float parser isn't optimal, probably the reason why we have fast-float now. So I wouldn't be surprised by the Rust team picking up that improvement after some time.

SWW13 avatar Aug 30 '22 10:08 SWW13