typst LSP panics on emacs

[Suya1671]

• Component:
  • [ ] VSCode Extension
  • [ ] VSCodium Extension
  • [x] LSP (used with other editor)
• Extension version: N/A
• LSP version: git (rev 7a9e8e0d5bf8ac24a6af44ec8a19d511c2373ea2), also tested on stable from nixpkgs
• OS version and name: NixOS Unstable

• [x] I am on the latest stable version of the extension/LSP.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.

Issue

I'm trying to get typst-lsp working with emacs (specifically DOOM emacs), however whenever I try to perform any lsp related actions (such as pressing enter to select a code completion) the server panics.

Doom emacs config code (using lsp-mode and typst-mode.el. typst-mode.el does not use any form of LSP):

(after! lsp-mode
  (add-to-list 'lsp-language-id-configuration
    '("\\.typ$" . "typst"))

  (lsp-register-client
    (make-lsp-client :new-connection (lsp-stdio-connection "typst-lsp")
                     :activation-fn (lsp-activate-on "typst")
                     :server-id 'typst-lsp)))

(use-package! typst-mode)

(add-hook 'typst-mode-local-vars-hook #'lsp!)
(add-hook 'typst--markup-mode-local-vars-hook #'lsp!)
(add-hook 'typst--code-mode-local-vars-hook #'lsp!)

Logs

thread 'main' panicked at 'slice index starts at 245 but ends at 212', /build/rustc-1.69.0-src/library/alloc/src/vec/mod.rs:2004:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Process typst-lsp stderr finished

[nvarner]

Can you follow the note from the logs and run with RUST_BACKTRACE=1 for more debugging info? I have some guesses, but that should narrow it down a lot.

[Suya1671]

Can you follow the note from the logs and run with RUST_BACKTRACE=1 for more debugging info? I have some guesses, but that should narrow it down a lot.

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/lsp_typst_boundary/mod.rs:101:58
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic
   3: typst_lsp::lsp_typst_boundary::lsp_to_typst::position_to_offset
   4: typst_lsp::server::lsp::<impl tower_lsp::LanguageServer for typst_lsp::server::TypstServer>::did_change::{{closure}}
   5: tower_lsp::jsonrpc::router::Router<S,E>::method::{{closure}}::{{closure}}::{{closure}}
   6: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
   7: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
   8: <core::pin::Pin<P> as core::future::future::Future>::poll
   9: <core::pin::Pin<P> as core::future::future::Future>::poll
  10: <tower_lsp::service::LspService<S> as tower_service::Service<tower_lsp::jsonrpc::Request>>::call::{{closure}}
  11: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  12: futures_util::stream::stream::StreamExt::poll_next_unpin
  13: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next
  14: <futures_util::stream::stream::filter_map::FilterMap<St,Fut,F> as futures_core::stream::Stream>::poll_next
  15: <futures_util::stream::stream::forward::Forward<St,Si,Item> as core::future::future::Future>::poll
  16: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  17: <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
  18: tokio::runtime::park::CachedParkThread::block_on
  19: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
  20: tokio::runtime::runtime::Runtime::block_on
  21: typst_lsp::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Process typst-lsp stderr finished
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/lsp_typst_boundary/mod.rs:101:58
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic
   3: typst_lsp::lsp_typst_boundary::lsp_to_typst::position_to_offset
   4: typst_lsp::server::lsp::<impl tower_lsp::LanguageServer for typst_lsp::server::TypstServer>::did_change::{{closure}}
   5: tower_lsp::jsonrpc::router::Router<S,E>::method::{{closure}}::{{closure}}::{{closure}}
   6: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
   7: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
   8: <core::pin::Pin<P> as core::future::future::Future>::poll
   9: <core::pin::Pin<P> as core::future::future::Future>::poll
  10: <tower_lsp::service::LspService<S> as tower_service::Service<tower_lsp::jsonrpc::Request>>::call::{{closure}}
  11: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  12: futures_util::stream::stream::StreamExt::poll_next_unpin
  13: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next
  14: <futures_util::stream::stream::filter_map::FilterMap<St,Fut,F> as futures_core::stream::Stream>::poll_next
  15: <futures_util::stream::stream::forward::Forward<St,Si,Item> as core::future::future::Future>::poll
  16: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  17: <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
  18: tokio::runtime::park::CachedParkThread::block_on
  19: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
  20: tokio::runtime::runtime::Runtime::block_on
  21: typst_lsp::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Process typst-lsp stderr finished

complete diff error now. Hmm

[nvarner]

This is almost certainly an encoding issue. See this article for technical details if you're curious.

What version of Emacs are you using?

You're probably working in a document that has emojis or Unicode math characters (e.g. 𝔸, not AA). Can you send that document if you're able, or otherwise try making an example document that triggers the same error?

[Enter-tainer]

What if we always choose utf16 https://github.com/nvarner/typst-lsp/blob/705098337453dc958ac8174e8ee75362c945fb22/src/config.rs#L213 ? @nvarner I see multiple emacs/nvim users report crashes.

[Enter-tainer]

VSCode is written in js and it should only accept utf16 while I guess nvim or other editors may accept encodings like utf8. Given that typst-lsp runs smoothly in vscode, i guess we can force on utf-16 and see if crashes still happens

[westernwontons]

I've found that using .unwrap_or(0) here: https://github.com/nvarner/typst-lsp/blob/0d5b9330a1d515fa47456e755f751ea4056282b8/src/lsp_typst_boundary.rs#L97-L98 makes the issue stop. I'm halfway through the tutorial and since the change I haven't noticed the error and everything seems to be functioning perfectly

[silverhikari]

getting this issue on vscode version 1.89.1 Linux running the default profile.