Parsing a very large tuple crashes the language server

[mbauman]

This file crashes the Julia language server:

julia> write("src/test.jl", repeat("0,",104300))
208600

Yes, that's a 200kB source file with nothing but 100k+ repetitions of 0, (and no, I didn't find this by fuzzing 🙈). Having this file in the workspace (included or not) crashes the language server with a StackOverflowError:

┌ Error: Some Julia code in the VS Code extension crashed
└ @ Main ~/.vscode/extensions/julialang.language-julia-1.66.2/scripts/error_handler.jl:15
ERROR: StackOverflowError:
Stacktrace:
 [1] emit(l::Tokenize.Lexers.Lexer{IOBuffer, Tokenize.Tokens.RawToken}, kind::Tokenize.Tokens.Kind, err::Tokenize.Tokens.TokenError)
   @ Tokenize.Lexers ~/.vscode/extensions/julialang.language-julia-1.66.2/scripts/packages/Tokenize/src/lexer.jl:276

This is on a macOS; I suppose the exact thresholds here may vary depending upon your system's stack limits.

[pfitzseb]

Turns out this works fine in JuliaSyntax (e.g. with JuliaSyntax.parseall(Expr,join(('0' for _ in 1:5043000), ','));). CSTParser is a bit more recursive and fails here for me:

julia> CSTParser.parse(repeat("0,",1043000));
ERROR: StackOverflowError:
Stacktrace:
  [1] optakessuffix
    @ Tokenize.Lexers ~/.julia/packages/Tokenize/JHBGP/src/utilities.jl:220 [inlined]
  [2] emit(l::Tokenize.Lexers.Lexer{IOBuffer, Tokenize.Tokens.RawToken}, kind::Tokenize.Tokens.Kind, err::Tokenize.Tokens.TokenError)
    @ Tokenize.Lexers ~/.julia/packages/Tokenize/JHBGP/src/lexer.jl:278
  [3] emit
    @ Tokenize.Lexers ~/.julia/packages/Tokenize/JHBGP/src/lexer.jl:277 [inlined]
  [4] lex_digit(l::Tokenize.Lexers.Lexer{IOBuffer, Tokenize.Tokens.RawToken}, kind::Tokenize.Tokens.Kind)
    @ Tokenize.Lexers ~/.julia/packages/Tokenize/JHBGP/src/lexer.jl:729
  [5] next_token(l::Tokenize.Lexers.Lexer{IOBuffer, Tokenize.Tokens.RawToken}, start::Bool)
    @ Tokenize.Lexers ~/.julia/packages/Tokenize/JHBGP/src/lexer.jl:390
  [6] next_token
    @ Tokenize.Lexers ~/.julia/packages/Tokenize/JHBGP/src/lexer.jl:315 [inlined]
  [7] next(ps::ParseState)
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/lexer.jl:102
  [8] parse_expression(ps::ParseState, esc_on_error::Bool)
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/packagedef.jl:48
  [9] parse_expression(ps::ParseState)
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/packagedef.jl:37
 [10] macro expansion
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/utils.jl:63 [inlined]
 [11] parse_tuple(ps::ParseState, ret::CSTParser.EXPR)
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/components/lists.jl:23
 [12] parse_compound(ps::ParseState, ret::CSTParser.EXPR)
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/packagedef.jl:135
 [13] parse_compound_recur(ps::ParseState, ret::CSTParser.EXPR) (repeats 79976 times)
    @ CSTParser ~/Documents/Git/julia-vscode/scripts/packages/CSTParser/src/packagedef.jl:92
    ```