JuliaSyntax.jl icon indicating copy to clipboard operation
JuliaSyntax.jl copied to clipboard

Better corpus testing

Open LilithHafner opened this issue 1 year ago • 2 comments

Whenever folks write new code in base or the stdlib that depends on JuliaSyntax bugfixes that have not been backported to flisp, CI will break on this repo. See #444, for an example where adding

https://github.com/JuliaLang/julia/blame/f2558c461c85be4220901f4c67cbce718ddc015b/test/core.jl#L8115

In https://github.com/JuliaLang/julia/pull/52228 caused CI to fail here.

Short term solution: stop testing against test/core.jl Long term solution: test against flisp using this corpus as of Julia v1.9 and do not include new code

LilithHafner avatar Jul 02 '24 15:07 LilithHafner

Another long term solution is to follow Julia's typo detection CI check: only flag parsing discrepancies that are not present in the previous commit of JuliaSyntax.

LilithHafner avatar Jul 02 '24 15:07 LilithHafner

I think that we should parse a massive corpus on the head and target commit of each PR and fail CI if there are any differences unless the PR is tagged with syntax change.

LilithHafner avatar Jul 03 '24 17:07 LilithHafner