useless error message in language server when both `tool.pyright` and `tool.basedpyright` are present in `pyproject.toml`

[gwww]

Using version 1.17 in neovim, I add the following to my pyproject.toml:

[tool.basedpyright]

reportMissingTypeStubs = false

The LSP reports the error:

LSP[basedpyright] Config file "/Users/glenn/Development/automation/elkm1/pyproject.toml" could not be parsed. Verify that format is correct.

As best as I can tell from the docs the format is correct. The snippet comes directly from the documentation.

Without the tool.basedpyright section I do not receive an error.

[DetachHead]

can you post the full contents of the pyproject.toml? or is that it?

[gwww]

Here is the complete pyproject.toml:

[project]
name = "elkm1-lib"
version = "2.2.7"
description = "Library for interacting with ElkM1 alarm/automation panel."
readme = "README.md"
requires-python = ">= 3.11"
authors = [{name = "Glenn Waters", email = "[email protected]"}]
classifiers = [
  'Development Status :: 5 - Production/Stable',
  "Intended Audience :: Developers",
  "Topic :: Home Automation",
  "License :: OSI Approved :: MIT License",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
]
dependencies = [
    "pyserial-asyncio-fast >= 0.14",
]

[tool.uv]
dev-dependencies = [
    "attrs >=24",
    "colorlog >=6.8",
    "mypy >=1.11",
    "pylint >=3.2",
    "urwid ==2.1.2",
    "pytest >=8.3",
    "pytest-asyncio >=0.23",
    "ruff >=0.6.1",
]

[project.urls]
Homepage = "https://github.com/gwww/elkm1"
Repository = "https://github.com/gwww/elkm1.git"
Issues = "https://github.com/gwww/elkm1/issues"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.pytest.ini_options]
asyncio_mode = "auto"

[tool.pylint."MESSAGES CONTROL"]
disable = [
    "format",  # Handled by black
    "abstract-method",  # With intro of async there are always methods missing
    "cyclic-import",  # Doesn't test if both import on load
    "duplicate-code",  # Unavoidable
    "inconsistent-return-statements",  # Doesn't handle raise
    "locally-disabled",  # It spams too much
    "not-context-manager",
    "too-few-public-methods",
    "too-many-ancestors",  # Too strict
    "too-many-arguments",
    "too-many-branches",
    "too-many-instance-attributes",
    "too-many-lines",
    "too-many-locals",
    "too-many-public-methods",
    "too-many-return-statements",
    "too-many-statements",
    "too-many-boolean-expressions",
    "unused-argument",  # Generic callbacks and setup methods create a lot of warnings
]
enable = [
    #"useless-suppression",  # temporarily every now and then to clean them up
    "use-symbolic-message-instead",
]

[tool.mypy]
python_version = "3.11"
ignore_missing_imports = true

[tool.pyright]
pythonVersion = "3.11"

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "F",    # Pyflakes
    "UP",   # pyupgrade
    "B",    # flake8-bugbear
    "SIM",  # flake8-simplify
    "I",    # isort
]

[tool.basedpyright]

reportMissingTypeStubs = false

The project is here: https://github.com/gwww/elkm1

[DetachHead]

the issue is that there's both a tool.pyright and tool.basedpyright section. you'll have to remove one of them.

not sure why the error message i wrote for this situation isn't being displayed

[gwww]

Thanks!

[DetachHead]

no problem, i'll keep this issue open because i want to investigate why you weren't getting the right error message

[ihabunek]

@DetachHead Why not allow both? We have a project where some engineers still use pyright and others switched to basedpyright. We'd like to be able to configure them independently.

Could you use basedpyright config if it exists and if not, then fall back to pyright?

[DetachHead]

one of the main goals of basedpyright is to ensure that the errors you see in your editor are consistent with other developers working on the same project and the errors reported in the CI, see https://github.com/microsoft/pylance-release/issues/5207#issuecomment-1912818825. this is something i feel very strongly about, and supporting that would go against this goal.

if different developers working on the same project are using different type checkers, one developer can commit and merge code that's full of type errors on another developer's IDE that they didn't see at all on their end. when this happens, people usually start to become alarm fatigued and ignore type errors completely because it's too difficult to determine which errors are your fault and which errors were introduced by someone else.

your project should ideally be configured such that everyone on the team is using the same type checker with the same config. your CI should also be set up with the same type checker and config, to ensure that no type errors get merged, and so that you can be confident that any error you see in your IDE is an actual problem in your branch that needs to be addressed otherwise the CI will fail.

[ihabunek]

your project should ideally be configured such that everyone on the team is using the same type checker with the same config.

I agree but here we are in an non-ideal world. :)

Thank you for the quick and elaborated response, and for your work on basedpyright.

[KotlinIsland]

I suppose there is a use-case such that basedpyright is not being used for type checking, but for other LSP features?

or maybe a case of gradual transition between the two

[woertsposzibllen4me]

I suppose there is a use-case such that basedpyright is not being used for type checking, but for other LSP features?

or maybe a case of gradual transition between the two

Well pyright is quite a bit faster than basedypright for auto-cmp, and it also probably will always have a thing or two that's more well rounded, the signature help window provided in neovim by pyright is better placed for example.

So I do use both LSP with basedpyright for diagnostics, over "interactive" usage. This is an argument FOR the exclusivity of having one tool supported in the toml only, as it's very easy to just turn off diagnostics for pyright at the langage server settings level..

But, I did spent quite some large amout of time triyng to figure out issues that had the variable of "only one toml works" mixed with them and complicating troubleshooting. Would it be possible for basedpyright to emit some kind of message when both [tool.pyright] and [tool.basedpyright] are in pyproject.toml ?

[DetachHead]

Well pyright is quite a bit faster than basedypright for auto-cmp, and it also probably will always have a thing or two that's more well rounded, the signature help window provided in neovim by pyright is better placed for example.

are you able to provide an example of these issues? i'm not aware of any performance differences between pyright and basedpyright, nor any differences in how signature help is displayed.

Would it be possible for basedpyright to emit some kind of message when both [tool.pyright] and [tool.basedpyright] are in pyproject.toml ?

yeah that's what's supposed to be happening, but for some reason the correct error message isn't being displayed, see https://github.com/DetachHead/basedpyright/issues/614#issuecomment-2309149599

[woertsposzibllen4me]

are you able to provide an example of these issues? i'm not aware of any performance differences between pyright and basedpyright, nor any differences in how signature help is displayed.

Sure:

1. cmp speed:

• based:

• pyright:

Here it's obviously not a matter of "actual coding speed" but of feel and enjoyment since we are only talking ms. But it is a super tiny project (400k chars). This said, beyond the "first request send" (particularly noticeable once for the functions, then once for the "constants" in demo) which has more significant obvious delay on "based", the "percentage wise diff" remains significant, I'd say it feels completely instantaneous on pyright vs a very brief delay on based... does that gap tend to widen or tighten with project scale, or stay the same, I have no idea.

(Obivously this is not gonna be as striking on those gifs at 16 fps here vs wezterm's 144)

2. signature positioning:

• based:

• notbased

It matches cursor position on pyright. This is only a concern of explicit calls made to vim.lsp.buf.signature_help() (Pressing a hotkey), signature showing up from trigger chars or when first entering insert mode in the function seems to usually match the cursor. Also only a concern for a certain signature help window size.

3. "case based sorting" cmp, whatever its called: (probably not sorting per se, but it looks like that's what's affected, with the only variable changing being the lsp here)

• based:

• pyright:

Very brief appartion of sc.. completions showing up is possible when typing SC with based before getting reordered. Not in pyright. (They might appear extremely briefely in pyright sometimes too, but for maybe 5 frames out of 144, cant tell on gifs.)

EDIT: note that in this example such "re-arranging" does eventually happen, but with different cmp setups (using blink over cmp, neovim plugins) I've observed occasional inconsistent behavior where case prioritization never eventually happens with basedpyright and SC just leads to sc, sci snippets.

• based: Might stay like this until restart/changes

Never seen that on pyright completions

[DetachHead]

thanks for the videos, very helpful!

can you see if the issues with completions go away when downgrading to basedpyright 1.27.1? if so, it's likely caused by deprecated tags on completions which were added in 1.28.0, in which case we should probably add an option to disable it.

as for the signature help positioning, that looks like a display issue in neovim. i'm not sure why it would only happen in basedpyright though, i haven't made any changes to signature help, and i don't think there's anything on the server side that can control its position.

[woertsposzibllen4me]

can you see if the issues with completions go away when downgrading to basedpyright 1.27.1? if so, it's likely caused by deprecated tags on completions which were added in 1.28.0, in which case we should probably add an option to disable it.

Tested it on 1.27.1 for a very short while only though:

1. So, performance wise, which I would have not really called an issue in the first place, I would def say that things feel a bit snappier yes altough not by a wide-shot, it's still not as fast as pyright which just feels like cocaine in comparison.

2. this:

• based:

seemed not to be a potential issue at all anymore,

thought this: showed up, but I could not compare or find another test-case with pyright: pyright does not provide SCM_RIGHTS as an Auto-Import symbol... so... just a whatever anecdote. (Pyright seems to provide less suggestions in autocomplete in general)

(pyright below):

3. Signature help window seems to be reliably properly placed according to cursor in 1.27.1:

[DetachHead]

So, performance wise, which I would have not really called an issue in the first place, I would def say that things feel a bit snappier yes altough not by a wide-shot, it's still not as fast as pyright which just feels like cocaine in comparison.

i have no idea then :( that's the only change we've made that i think could cause such an issue. i guess some other things to try:

• try older versions and see if you can pinpoint what release introduced the issue
• set typeCheckingMode to "standard". perhaps some of the stricter checks are slowing things down?

as for the other issues, i'm not really sure about those either. i've never seen a ~ show up in a completion before. it looks like that might be coming from your editor rather than (based)pyright. i'm guessing they show up for longer while basedpyright loads the completions

(Pyright seems to provide less suggestions in autocomplete in general)

yeah one of the improvements we've made is to cache the whole standard library for completions. otherwise suggestions from most modules won't appear unless they've already been imported before. it's possible this could also be causing some performance issues but i would think only on the first completion, any subsequent completions should be unaffected.

that feature was added in 1.17.4 (#655) so maybe see if 1.17.3 behaves any different?

[woertsposzibllen4me]

yeah one of the improvements we've made is to cache the whole standard library for completions. otherwise suggestions from most modules won't appear unless they've already been imported before. it's possible this could also be causing some performance issues but i would think only on the first completion, any subsequent completions should be unaffected.

that feature was added in 1.17.4 (#655) so maybe see if 1.17.3 behaves any different?

Yes that's the culprit then most likely. Reverting to 1.17.3 brings back completion speed up to par with regular pyright. On my machine, which is pretty fast (Ryzen 5900x), with the very small inaccurate tests I've made we are talking about a pretty reliable obversation of an average for completion time showing up going from about 120-140ms to 45-65ms, excluding the "first ever" completion which might take about a whole second more. (This is reset after a while tho isn't it ? Does the server go to sleep under certain condition making that inital loadup happen again ? I feel like I've observed this).

So this would mean likely that this ~80ms difference is caused by the extra caching and not performance difference between the servers, and that it would then not be a gap that would widen along with project scale: it could be conceived more of a flat tax to pay for the luxury of auto importing a stdlib the first time in a project. This is, in my (often controversional) opinion though, a pretty hefty price to pay. That's especially relevant for my specific use case argument, since having basedpyright work alongside pyright would provide a very pleasant and easy experience of resolving first time occurences of unimported standardlib modules via the code actions provided by basedpyright since 1.17.4 as well instead... which then allows the 'auto-imports' to work as usual for next time imports in the same project.

But obviously then, if it's just about the only reason to have two whole LSP running, it would feel at least quite sensible to want to ditch pyright and have basedpyrighy only 😅 . Maybe an option to just turn this off would make a reasonable amount of sense ? That would require 'completion auto-imports' to be able to be de-coupled from 'code action imports' then ..

Ye I wouldn't really discuss the ~ snippets code further there are too many moving parts and I've no idea what I am observing let alone what could be happening

[DetachHead]

i've opened #1345 to track this performance issue since it's not really related to this issue. thanks for the help narrowing it down