JuliaSyntax.jl icon indicating copy to clipboard operation
JuliaSyntax.jl copied to clipboard

eliminate dependency on Base.is_id_char and julia-specific utf8proc

Open stevengj opened this issue 2 years ago • 2 comments

As discussed in #372, in order to make the JuliaSyntax parsing independent of the version of Julia in which it runs, it needs to:

  1. Add a dependency on utf8proc_jll, rather than using the utf8proc bundled with Julia. This is important because the supported version of Unicode (for character classification) depends on utf8proc.
  2. Re-implement Base.is_id_start_char and Base.is_id_char to identify which characters can appear in identifiers (or technically, which characters can begin graphemes, now that #372 is merged … this could simplify the rules slightly since we no longer need to explicitly check for combining marks).

stevengj avatar Nov 01 '23 12:11 stevengj

Thanks for outlining what needs to be done here :)

Just wondering - do you think you'll have bandwidth to work on this yourself or should I or someone else take it on? I feel like tackling it relatively soon because I'd like to be able to bump JuliaSyntax version which is used in Base, even on Julia-1.10.x without introducing new features. And I belated realized that #372 breaks the ability to do that.

c42f avatar Nov 01 '23 23:11 c42f

I'm working on it.

stevengj avatar Nov 04 '23 21:11 stevengj