Python syntax highlighting fails without window reload

[tylerfxn]

Getting python syntax highlighting to apply to newly written code is requiring a window reload.

• Remote SSH
• Cursor Pyright + ms-python Python
• Amazon Linux 2 (Red Hat)

If I type anything new, eg, print() and save the file, the following is logged:

2024-01-27 21:26:27.146 [error] [ms-python.python] provider FAILED
2024-01-27 21:26:27.146 [error] Error: No semantic tokens provided/needed for file:///home/ec2-user/my/python/file.py
	at /home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2308663
	at re (/home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2308957)
	at /home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2303741
	at Immediate.<anonymous> (/home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2303761)
	at process.processImmediate (node:internal/timers:476:21)

See the syntax highlighting above not applied to the last two prints.

[tylerfxn]

2024-01-27 22:42:27.921 [info] [Error - 10:42:27 PM] Request textDocument/semanticTokens/full/delta failed.

[Sanger2000]

With Cursor Pyright, you should be using anysphere.python (rather than ms-python.python). Can help you figure this out via email if you send details to [email protected]

[tylerfxn]

Thanks. Where is anysphere.python supposed to come from? It's not mentioned in any docs, across google, or in the extensions search. Taking this over email won't help others lacking the same information.

[arkaydeus]

In addition isort and Black Formatter rely on ms-python (preventing it being uninstalled).

Is there a decent solution here where we can still sort imports and format, while using Cursor Python?

Is there any reason why this hasn't been documented and explained?

[tylerfxn]

I'm guessing anysphere.python is an unallowed fork of ms.python

[lukaemon]

Thanks. Where is anysphere.python supposed to come from? It's not mentioned in any docs, across google, or in the extensions search. Taking this over email won't help others lacking the same information.

Did you find anysphere.python?

[arkaydeus]

Is there a solution to this. Python still looks completely broken in Cursor. One of the most popular languages out there completely unusable in Cursor. Surely there must be a solution?

[arkaydeus]

To summarise, cursor + cursor pyright + python doesn't work as syntax highlighting fails and that's a core part of the language server. It's basically failing to pickup tokens. Something is broken which means you can't really use the Python language in Cursor at the moment.

[arkaydeus]

Great. But please could you explain how you fixed it as it still doesn't work for other people.

[navels]

I would also like to know the magical incantation required to get syntax highlighting working.

[navels]

Nothing at the client side. They pushed something in their backend that solved my problems.

lol it was probably if username == "lukaemon" . . .

[rian-dolphin]

Any update on this?

[jkryanchou]

Any update to this issue ? I have met the same issue. While using the pylance language server won't raise the exception

2024-09-01 12:46:25.111 [info] (Client) Pylance async client (2024.8.1) started with python extension (2024.12.3)
2024-09-01 12:46:25.357 [info] [Info  - 12:46:25 PM] (592) Pylance language server 2024.8.1 (pyright version 1.1.373, commit ee424479) starting
2024-09-01 12:46:25.357 [info] [Info  - 12:46:25 PM] (592) Server root directory: file:///root/.cursor-server/extensions/ms-python.vscode-pylance-2024.8.1/dist
2024-09-01 12:46:25.362 [info] [Info  - 12:46:25 PM] (592) Starting service instance "icc-cloud-audit"
2024-09-01 12:46:25.386 [info] [Info  - 12:46:25 PM] (592) Setting pythonPath for service "icc-cloud-audit": "/workspaces/icc-cloud-audit/.venv/bin/python"
2024-09-01 12:46:25.386 [info] [Info  - 12:46:25 PM] (592) Setting environmentName for service "project": "3.12.5 (.venv venv)"
2024-09-01 12:46:25.386 [info] [Info  - 12:46:25 PM] (592) Loading pyproject.toml file at /workspaces/<project>/pyproject.toml
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) Pyproject file "/workspaces/<project>/pyproject.toml" has no "[tool.pyright]" section.
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) No include entries specified; assuming /workspaces/<project>
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) Auto-excluding **/node_modules
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) Auto-excluding **/__pycache__
2024-09-01 12:46:25.391 [info] [Info  - 12:46:25 PM] (592) Auto-excluding **/.*
2024-09-01 12:46:25.416 [info] [Info  - 12:46:25 PM] (592) Assuming Python version 3.12.5.final.0
2024-09-01 12:46:25.632 [info] [Info  - 12:46:25 PM] (592) Found 18 source files
2024-09-01 12:46:25.634 [info] [Info  - 12:46:25 PM] (592) Background analysis(1) root directory: file:///root/.cursor-server/extensions/ms-python.vscode-pylance-2024.8.1/dist
2024-09-01 12:46:25.634 [info] [Info  - 12:46:25 PM] (592) Background analysis(1) started
2024-09-01 12:46:33.402 [info] [Info  - 12:46:33 PM] (592) Indexer background runner(2) root directory: file:///root/.cursor-server/extensions/ms-python.vscode-pylance-2024.8.1/dist (index)
2024-09-01 12:46:33.402 [info] [Info  - 12:46:33 PM] (592) Indexing(2) started
2024-09-01 12:46:33.939 [info] [Info  - 12:46:33 PM] (592) scanned(2) 197 files over 1 exec env
2024-09-01 12:46:34.212 [info] [Info  - 12:46:34 PM] (592) indexed(2) 149 files over 1 exec env
2024-09-01 12:46:34.288 [info] [Info  - 12:46:34 PM] (592) Indexing finished(2).

While using the pyright raise exception as unexpected.

2024-09-01 13:06:27.722 [info] [Info  - 1:06:27 PM] Pyright language server 1.1.325 starting
2024-09-01 13:06:27.722 [info] [Info  - 1:06:27 PM] Server root directory: /root/.cursor-server/extensions/anysphere.pyright-1.1.327-universal/dist
2024-09-01 13:06:27.724 [info] [Info  - 1:06:27 PM] Starting service instance "x-insight"
2024-09-01 13:06:27.745 [info] [Info  - 1:06:27 PM] Setting pythonPath for service "x-insight": "/venv/bin/python"
2024-09-01 13:06:27.752 [info] [Info  - 1:06:27 PM] Loading pyproject.toml file at /workspaces/x-insight/pyproject.toml
2024-09-01 13:06:27.756 [info] [Info  - 1:06:27 PM] Pyproject file "/workspaces/x-insight/pyproject.toml" has no "[tool.pyright]" section.
2024-09-01 13:06:27.756 [info] [Warn  - 1:06:27 PM] stubPath /workspaces/x-insight/typings is not a valid directory.
2024-09-01 13:06:27.776 [info] [Info  - 1:06:27 PM] Assuming Python version 3.12
2024-09-01 13:06:27.875 [info] [Info  - 1:06:27 PM] Auto-excluding /workspaces/x-insight/venv3
2024-09-01 13:06:27.884 [info] [Info  - 1:06:27 PM] Found 61 source files
2024-09-01 13:06:53.425 [info] [Error - 1:06:53 PM] Request textDocument/semanticTokens/full/delta failed.
2024-09-01 13:06:53.425 [info]   Message: No semantic tokens provided/needed for file:///workspaces/x-insight/xinsight/telegram_bot.py
  Code: -32603 
2024-09-01 13:06:54.571 [info] [Error - 1:06:54 PM] Request textDocument/semanticTokens/full/delta failed.
2024-09-01 13:06:54.571 [info]   Message: No semantic tokens provided/needed for file:///workspaces/x-insight/xinsight/telegram_bot.py
  Code: -32603 
2024-09-01 13:06:57.776 [info] [Error - 1:06:57 PM] Request textDocument/semanticTokens/full/delta failed.
2024-09-01 13:06:57.776 [info]   Message: No semantic tokens provided/needed for file:///workspaces/x-insight/xinsight/telegram_bot.py
  Code: -32603 
2024-09-01 13:06:59.416 [info] [Error - 1:06:59 PM] Request textDocument/semanticTokens/full/delta failed

[rian-dolphin]

I seem to have got it working though I'm not quite sure what did it. I did the following:

Uninstalled cursor by doing the following (on mac):

1. Quit Cursor if it's running.
2. Go to the Applications folder and move the Cursor app to the Trash.
3. Clean up associated files:
   • Delete any folders or files related to Cursor in these directories: ~/Library/Application Support/ ~/Library/Caches/ ~/Library/Preferences/ ~/Library/Saved Application State/
4. Empty the Trash.

Reinstalled cursor and selected the option to continue from scratch without any of my vscode settings.

Then the highlighting was working, BUT when I changed to the Dark Modern theme it stopped working again. I then switched back to Anysphere Dark and back to Dark Modern again and it was working.

As I said, I'm not sure what did it but hopefully it can help someone else.

[arkaydeus]

I'm STILL getting this issue when using the Dark+ theme.

How to recreate:

Fresh install of cursor. Open a Python project Add new import (not highlighted) Use new import (not highlighted)

Note that if you close the file and open it again, it then correctly highlights:

This is incredibly frustrating and has lasted over 6 months now. PLEASE can someone provide advice on how they were able to solve this?

[arkaydeus]

BUT when I changed to the Dark Modern theme it stopped working again.

Can you confirm if you got it working persistently? For me when I restart, it correctly highlights when the file is first open or the window is refreshed, but it doesn't carry on working when I type new code. It looks like the deltas aren't being passed to the language server.

[arkaydeus]

OK, I've made some progress on this.

Currently running MacOS. I installed a new profile (user) on my machine and strangely all worked fine. What I noticed though, is that different versions of the extensions are installed.

Compare the two. The darwin-arm64 versions work fine. The universal versions don't and keep running into issues. They're also older.

Following up and checking the two extensions.json files you can see the working version on the right and the broken version on the left.

On the right the targetPlatform is ‘darwin-arm64’ which is correct. On the left it’s ‘universal’ and then the version is 2024.5.1 and not 2024.12.3

This is incredibly frustrating. Same machine. Same OS. Just different user profile.

Would be really interested to see how I can force a particular architecture for extensions.

[0x-chad]

+1 on this issue

[ilbertt]

+1

Happening for me too, really annoying considering the price of the subscription and the fact that bug reduces a lot the productivity

[arkaydeus]

Have you guys tried migrating users? Really annoying that you have to do that, but it did allow me to install arm64 versions by default which didn't break.

[arkaydeus]

For anyone who finds this issue, has an M series Mac and isn't sure how to fix it short term, got to the .cursor folder for your user and the extensions folder. IF you find you have python extensions that end in -universal, then your system is installing the wrong versions. You need -darwin-arm64 versions.

While I couldn't find a way to force architecture for extensions, creating a new user on the machine and then installing Cursor in that user profile did fix the problem. Even a complete clean of the old user and re-install DIDNT fix it.

So if you can stomach migrating to a new user profile, it's likely you will be able to fix this.

Would love to know if there's a way to force architecture.

[ilbertt]

I will never create a new profile on my computer just to fresh-install Cursor and make it work with the extensions. It's really annoying that the cursor team is not taking care of this issue, considering how much the product costs.

I even emailed them explaining my issue, without ever getting a reply. I have not subscribed to the yearly plan only because of this issue. Now that the monthly renewal is getting closer, I will definitely consider NOT renewing my subscription and go back to VSCode + Codeium, which never gave me issues and never significantly reduced my productivity like this bug does.

I hope Cursor will start listening to their paying customers at some point.

[arkaydeus]

There's probably a way to force architecture for plugins. Does anyone know how to do this?

[simonryf]

I finally got it working, though this is by far not ideal. No idea what happens to auto update etc. Also I have not yet worked intensively with this. So if I run into further errors I will post again here. However syntax highlighting works right a way and not only after restarting python language server.

@arkaydeus Kudos to you! I couldn't do it without your info here. However switching to another user in macos didn't solve it for me. So I tried and tried and tried...

System Info

❯ sw_vers
ProductName:        macOS
ProductVersion:     15.0.1
BuildVersion:       24A348

~
❯ cursor -v
0.42.3
949de58bd3d85d530972cac2dffc4feb9eee1e40
arm64

1. Backup current extensions / start from scratch

# Backup all Extensions
mv ~/.cursor/extensions /tmp/
mkdir ~/.cursor/extensions

2. Download python for darwin-arm64 via Browser and drag the .vsxi in cursor extensions UI

(If you ask how I got to this link, I had a gazillion browser tabs open and from somewhere i got a similar link and adapted at according to arkaydeus screenshot above)

https://marketplace.visualstudio.com/_apis/public/gallery/publishers/ms-python/vsextensions/python/2024.12.3/vspackage?targetPlatform=darwin-arm64

3. Check

cat ~/.cursor/extensions/extensions.json

I have now "targetPlatform": "darwin-arm64" for ms-python

4. Download pylance via Browser and also dragged it into cursor extensions UI

https://marketplace.visualstudio.com/_apis/public/gallery/publishers/ms-python/vsextensions/vscode-pylance/2024.8.1/vspackage

-> This automatically installed also Cursor Pyright

Now my extensions.json looks a bit weird, but it works for me.

[
  {
    "identifier": {
      "id": "ms-python.debugpy",
      "uuid": "4bd5d2c9-9d65-401a-b0b2-7498d9f17615"
    },
    "version": "2024.6.0",
    "location": {
      "$mid": 1,
      "path": "/Users/sryf/.cursor/extensions/ms-python.debugpy-2024.6.0-darwin-arm64",
      "scheme": "file"
    },
    "relativeLocation": "ms-python.debugpy-2024.6.0-darwin-arm64",
    "metadata": {
      "installedTimestamp": 1729945269792,
      "pinned": true,
      "source": "gallery",
      "id": "4bd5d2c9-9d65-401a-b0b2-7498d9f17615",
      "publisherId": "998b010b-e2af-44a5-a6cd-0b5fd3b9b6f8",
      "publisherDisplayName": "ms-python",
      "targetPlatform": "darwin-arm64",
      "updated": false,
      "isPreReleaseVersion": false,
      "hasPreReleaseVersion": false
    }
  },
  {
    "identifier": {
      "id": "ms-python.python"
    },
    "version": "2024.12.3",
    "location": {
      "$mid": 1,
      "path": "/Users/sryf/.cursor/extensions/ms-python.python-2024.12.3",
      "scheme": "file"
    },
    "relativeLocation": "ms-python.python-2024.12.3",
    "metadata": {
      "installedTimestamp": 1729945269791,
      "source": "vsix",
      "id": "f1f59ae4-9318-4f3c-a9b5-81b2eaa5f8a5",
      "publisherDisplayName": "ms-python",
      "publisherId": "998b010b-e2af-44a5-a6cd-0b5fd3b9b6f8",
      "isPreReleaseVersion": false,
      "hasPreReleaseVersion": false
    }
  },
  {
    "identifier": {
      "id": "anysphere.pyright",
      "uuid": "d990d7f1-60b6-4234-91a7-d146a42bb230"
    },
    "version": "1.1.327",
    "location": {
      "$mid": 1,
      "path": "/Users/sryf/.cursor/extensions/anysphere.pyright-1.1.327-universal",
      "scheme": "file"
    },
    "relativeLocation": "anysphere.pyright-1.1.327-universal",
    "metadata": {
      "installedTimestamp": 1729945578486,
      "pinned": true,
      "source": "gallery",
      "id": "d990d7f1-60b6-4234-91a7-d146a42bb230",
      "publisherId": "963cf79c-e108-4418-8858-b88cc178c014",
      "publisherDisplayName": "Anysphere",
      "targetPlatform": "universal",
      "updated": false,
      "isPreReleaseVersion": false,
      "hasPreReleaseVersion": false
    }
  },
  {
    "identifier": {
      "id": "ms-python.vscode-pylance"
    },
    "version": "2024.8.1",
    "location": {
      "$mid": 1,
      "path": "/Users/sryf/.cursor/extensions/ms-python.vscode-pylance-2024.8.1",
      "scheme": "file"
    },
    "relativeLocation": "ms-python.vscode-pylance-2024.8.1",
    "metadata": {
      "installedTimestamp": 1729945578486,
      "pinned": true,
      "source": "vsix"
    }
  }
]

5. Install Color -, File Icon - & Product Icon Themes via Cursor Extensions UI

I like:

• One Dark Pro
• Material Icon Theme
• Material Product Icons

(But that's up to your preference :-)

[zaradana]

What worked for me wast uninstalling and installing "Cursor Pyright" extention from cursor. I am using Dark Modern theme, not sure if that makes a difference.

[arkaydeus]

Really hope these stick. My issue before I switched user was that after a while it would update and download the universal one over the top of arm64 version.

Save all your working ones in a backup folder so you can refresh that if it overwrites.

[joeblackwaslike]

Nothing works for me, the truth is sadly cursor does not support python and theres no indication from the devs whether it ever will. Sadly, this makes cursor a non starter for me at this time, but I would love to try again in the future if they decide to start caring about python.

[ilbertt]

Nothing works for me, the truth is sadly cursor does not support python and theres no indication from the devs whether it ever will. Sadly, this makes cursor a non starter for me at this time, but I would love to try again in the future if they decide to start caring about python.

Definitely agree with you. I'll also probably not renew the subscription the next month

[xdxindustries]

@simonryf Thank you, can confirm this is the thing that finally fixed it for me. Thank you very much!

[zjy110-openai]

@simonryf this worked for me as well, thank you so much!