bug: LSP provider doesn't handle multibyte strings

[kiyoon]

Describe the bug

Note that the line without Korean characters (id_) is highlighted correctly.

The treesitter and regex provider works correctly.

To Reproduce

Install basedpyright and vim-illuminate. Use the example code.

class 클래스(Base):  # noqa: N801
    __tablename__ = "Table"
    id_: Mapped[int] = mapped_column(primary_key=True)
    클래스ID: Mapped[int] = mapped_column()
    한글: Mapped[str] = mapped_column()
    한글2: Mapped[str] = mapped_column()
    테스트: Mapped[int] = mapped_column()

nvim -u repro.lua example.py

vim.env.LAZY_STDPATH = ".repro"
load(vim.fn.system("curl -s https://raw.githubusercontent.com/folke/lazy.nvim/main/bootstrap.lua"))()

require("lazy.minit").repro({
	spec = {
		{
			"RRethy/vim-illuminate",
			config = function()
				require("illuminate").configure({
					providers = {
						"lsp",
						"treesitter",
						"regex",
					},
				})
			end,
		},
		{
			"neovim/nvim-lspconfig",
		},
	},
})

vim.lsp.enable("basedpyright")

Output from :IlluminateDebug

buf_should_illuminate 1 true
config {
  case_insensitive_regex = false,
  delay = 100,
  disable_keymaps = false,
  filetype_overrides = {},
  filetypes_allowlist = {},
  filetypes_denylist = { "dirbuf", "dirvish", "fugitive" },
  large_file_cutoff = 10000,
  min_count_to_highlight = 1,
  modes_allowlist = {},
  modes_denylist = {},
  providers = { "lsp", "treesitter", "regex" },
  providers_regex_syntax_allowlist = {},
  providers_regex_syntax_denylist = {},
  under_cursor = true
}
started true
provider table: 0x01035f42e0 lsp
`termguicolors` true

[scc02]

Similar issue