Index out of bounds in `analysis.documentPositionContext` when using VSCode Live Share

[jceipek]

Thanks for the awesome tool; it's made it a lot easier for me to get started with zig!

Unfortunately, while learning Zig together with friends in a different country, I discovered a reproducible index-out-of-bounds panic when using zls and zig master in conjunction with VSCode's official Live Share extension (https://marketplace.visualstudio.com/items?itemName=MS-vsliveshare.vsliveshare-pack). Live Share's architecture uses a very simple dumb terminal model, where the host machine is in charge of running extensions and responding to input from remote participants who can see and interact with the code, so it shouldn't generally require any extra work to support it as far as I can tell.

When I start a Live Share session and connect to it (via the same machine or a different one), syntax highlighting doesn't work on the client(s). If a client starts typing something, the host's zls server reliably panics with the following trace:

[debug-main] Took 1ms to process method textDocument/didChange
thread 6474589 panic: index out of bounds
/Users/julianceipek/Development/zls-mac/zls/src/analysis.zig:1541:48: 0x10e3e4468 in analysis.documentPositionContext (zls)
    var tokenizer = std.zig.Tokenizer.init(line[0..doc_position.line_index]);
                                               ^
/Users/julianceipek/Development/zls-mac/zls/src/main.zig:1414:61: 0x10e3cff77 in completionHandler (zls)
    const pos_context = try analysis.documentPositionContext(arena, handle.document, doc_position);
                                                            ^
/Users/julianceipek/Development/zls-mac/zls/src/main.zig:1675:53: 0x10e3a454f in processJsonRpc (zls)
                    done = extractErr(method_info[2](arena, id, request_obj, config));
                                                    ^
/Users/julianceipek/Development/zls-mac/zls/src/main.zig:1916:27: 0x10e39d0f3 in main (zls)
        try processJsonRpc(&arena, &json_parser, buf, config);
                          ^
/Users/julianceipek/Downloads/zig-macos-x86_64-0.8.0-dev.2133+ad33e3483/lib/zig/std/start.zig:420:37: 0x10e4c8cb8 in std.start.callMain (zls)
            const result = root.main() catch |err| {
                                    ^
/Users/julianceipek/Downloads/zig-macos-x86_64-0.8.0-dev.2133+ad33e3483/lib/zig/std/start.zig:362:12: 0x10e3a6267 in std.start.callMainWithArgs (zls)
    return @call(.{ .modifier = .always_inline }, callMain, .{});
           ^
/Users/julianceipek/Downloads/zig-macos-x86_64-0.8.0-dev.2133+ad33e3483/lib/zig/std/start.zig:332:12: 0x10e3a61a5 in std.start.main (zls)
    return @call(.{ .modifier = .always_inline }, callMainWithArgs, .{ @intCast(usize, c_argc), c_argv, envp });
           ^
???:?:?: 0x7fff20596620 in ??? (???)
???:?:?: 0x0 in ??? (???)
[Info  - 11:11:16 PM] Connection to server got closed. Server will restart.
[Error - 11:11:16 PM] Request textDocument/completion failed.
Error: Connection got disposed.
	at Object.dispose (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/connection.js:1044:27)
	at Object.dispose (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/common/client.js:73:35)
	at LanguageClient.handleConnectionClosed (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/common/client.js:2352:42)
	at LanguageClient.handleConnectionClosed (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/node/main.js:155:15)
	at closeHandler (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/common/client.js:2339:18)
	at CallbackList.invoke (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:55:39)
	at Emitter.fire (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:117:36)
	at closeHandler (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/connection.js:256:26)
	at CallbackList.invoke (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:55:39)
	at Emitter.fire (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:117:36)
	at StreamMessageReader.fireClose (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/messageReader.js:40:27)
	at Socket.<anonymous> (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/messageReader.js:125:42)
	at Socket.emit (events.js:327:22)
	at Socket.EventEmitter.emit (domain.js:483:12)
	at Pipe.<anonymous> (net.js:674:12)
[Error - 11:11:16 PM] Request textDocument/semanticTokens/full failed.
Error: Connection got disposed.
	at Object.dispose (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/connection.js:1044:27)
	at Object.dispose (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/common/client.js:73:35)
	at LanguageClient.handleConnectionClosed (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/common/client.js:2352:42)
	at LanguageClient.handleConnectionClosed (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/node/main.js:155:15)
	at closeHandler (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-languageclient/lib/common/client.js:2339:18)
	at CallbackList.invoke (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:55:39)
	at Emitter.fire (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:117:36)
	at closeHandler (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/connection.js:256:26)
	at CallbackList.invoke (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:55:39)
	at Emitter.fire (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/events.js:117:36)
	at StreamMessageReader.fireClose (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/messageReader.js:40:27)
	at Socket.<anonymous> (/Users/julianceipek/.vscode/extensions/augusterame.zls-vscode-1.0.4/node_modules/vscode-jsonrpc/lib/common/messageReader.js:125:42)
	at Socket.emit (events.js:327:22)
	at Socket.EventEmitter.emit (domain.js:483:12)
	at Pipe.<anonymous> (net.js:674:12)

To reproduce:

1. Create a new zig project with zig init-exe
2. Open the folder containing build.zig in VSCode with the zls-vscode extension installed and enabled
3. Make sure https://marketplace.visualstudio.com/items?itemName=MS-vsliveshare.vsliveshare-pack is installed and enabled
4. Start a Live Share session by clicking the Live Share button at the bottom of the VSCode window, which copies a URL to the clipboard
5. Open the Live Share link in a browser (can be on the same machine for testing or a different one, which is where I first experienced the problem), which opens a client VSCode session
6. Navigate to main.zig in the client session. Note that syntax highlighting is not present.
7. Type std. inside the main function
8. The zls executable panics on the host machine and the above trace shows up inside VSCode's Output tab under Zig Language Server

Based on the trace, I suspect that the source of the problem lies with zls and not with zls-vscode or vsliveshare. However, there have been rare cases of issues with LiveShare and language extensions. See, for example https://github.com/Dart-Code/Dart-Code/issues/2269 and the associated https://github.com/MicrosoftDocs/live-share/issues/3509. I'm super new to zig, but please let me know if there's any way I can help track down the cause of this bug.