vscode-copilot-release icon indicating copy to clipboard operation
vscode-copilot-release copied to clipboard

GitHub Copilot Chat unable to generate commit messages due to content exclusion rules

Open liby opened this issue 1 year ago • 39 comments

I have been using GitHub Copilot Chat for generating commit messages, but recently, this feature has stopped working. Regardless of the file changes, I receive the following message: "Cannot generate a commit message because the changes were excluded from the context due to content exclusion rules."

  • VS Code Version: 1.89.1
  • Commit: dc96b837cf6bb4af9cd736aa3af08cf8279f7685
  • Date: 2024-05-07T05:14:24.611Z (1 week ago)
  • Electron: 28.2.8
  • ElectronBuildId: 27744544
  • Chromium: 120.0.6099.291
  • Node.js: 18.18.2
  • V8: 12.0.267.19-electron.0
  • OS: Darwin x64 22.5.0
  • GitHub Copilot Chat Version: v0.15.2024043005 (pre-release)

Steps to Reproduce:

  1. Open any project in VS Code.
  2. Make some changes to the files.
  3. Attempt to generate a commit message using GitHub Copilot Chat.
  4. Observe the error message: "Cannot generate a commit message because the changes were excluded from the context due to content exclusion rules."
image

liby avatar May 21 '24 02:05 liby

I also use Copilot to generate commit messages and can confirm that the feature no longer works. However, I don't get the same error message as you, but keep getting an old commit message even though the changes have long since been committed.

  • Version: 1.89.1 (user setup)
  • Commit: dc96b837cf6bb4af9cd736aa3af08cf8279f7685
  • Date: 2024-05-07T05:13:33.891Z
  • Electron: 28.2.8
  • ElectronBuildId: 27744544
  • Chromium: 120.0.6099.291
  • Node.js: 18.18.2
  • V8: 12.0.267.19-electron.0
  • OS: Windows_NT x64 10.0.19045
  • GitHub Copilot Chat: 0.15.2

AndreasKleineberg avatar May 22 '24 20:05 AndreasKleineberg

@liby - does it work if you highlight the files under changes and select "stage"?

canuckjacq avatar May 24 '24 15:05 canuckjacq

@liby - does it work if you highlight the files under changes and select "stage"?

image

liby avatar May 27 '24 02:05 liby

Can you share your log file please. Via CMD/CTRL + SHIFT+ U -> GitHub Copilot Chat

lramos15 avatar May 28 '24 13:05 lramos15

Same issue here, I can type a commit message and commit and then commits after work fine

joeybab3 avatar May 28 '24 21:05 joeybab3

Can you share your log file please. Via CMD/CTRL + SHIFT+ U -> GitHub Copilot Chat

It looks very similar to this issue, not sure if it's related.

2024-05-30 13:10:47.440 [info] [FetcherService] Using Helix fetcher
2024-05-30 13:10:47.440 [info] [gitExtensionService] Initializing Git extension service.
2024-05-30 13:10:47.440 [info] [gitExtensionService] Successfully activated the vscode.git extension.
2024-05-30 13:10:47.440 [info] [gitExtensionService] Enablement state of the vscode.git extension: true.
2024-05-30 13:10:47.440 [info] [gitExtensionService] Successfully registered Git commit message provider.
2024-05-30 13:10:47.967 [info] [auth] Logged in as liby
2024-05-30 13:10:48.651 [info] [chat] copilot token chat_enabled: true
2024-05-30 13:10:48.651 [info] [chat] Registration of interactive providers failed:,Error: Extension 'GitHub.copilot-chat' CANNOT use API proposal: aiTextSearchProvider.
Its package.json#enabledApiProposals-property declares: interactive, terminalDataWriteEvent, terminalExecuteCommandEvent, terminalSelection, terminalQuickFixProvider, chatParticipant, chatParticipantAdditions, defaultChatParticipant, chatVariableResolver, chatProvider, mappedEditsProvider, aiRelatedInformation, codeActionAI, findTextInFiles, textSearchProvider, contribSourceControlInputBoxMenu, newSymbolNamesProvider, findFiles2, extensionsAny, authLearnMore, testObserver but NOT aiTextSearchProvider.
 The missing proposal MUST be added and you must start in extension development mode or use the following command line switch: --enable-proposed-api GitHub.copilot-chat
2024-05-30 13:10:48.651 [info] [githubTitleAndDescriptionProvider] Initializing GitHub PR title and description provider provider.
2024-05-30 13:10:48.651 [info] [githubTitleAndDescriptionProvider] GitHub.vscode-pull-request-github extension is not yet activated.
2024-05-30 13:10:48.655 [info] [auth] Got Copilot token for liby
2024-05-30 13:10:50.181 [info] [RemoteAgents] Invalid remote agent response: 404 page not found
 (SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5))

liby avatar May 30 '24 05:05 liby

@liby Can you ask a question in the panel chat about this file? Like typing /explain does that answer or also give issues regarding content exclusions

lramos15 avatar May 30 '24 13:05 lramos15

Same issue here. Just started happening today.

JamesDBartlett3 avatar May 30 '24 16:05 JamesDBartlett3

@liby Can you ask a question in the panel chat about this file? Like typing /explain does that answer or also give issues regarding content exclusions

image

liby avatar May 30 '24 16:05 liby

Do you see the same issue if you try VS Code insiders + pre-release https://code.visualstudio.com/insiders/

Sory for all the questions, we're just having issues attempting to repro this

lramos15 avatar May 30 '24 17:05 lramos15

Do you see the same issue if you try VS Code insiders + pre-release code.visualstudio.com/insiders

Sory for all the questions, we're just having issues attempting to repro this

By installing the VS Code insiders(without any personal configuration) and only installing 2 extensions (GitHub Copilot and GitHub Copilot Chat), I still encounter the same issue.

image

liby avatar May 30 '24 17:05 liby

I have the exact same issue. It started in march/april. Here's my reproduction with the latest insiders and only the copilot extensions installed.

afb-requests-request-form-autosuggest-2024-03-25-5

Version: 1.90.0-insider
Commit: ca688da9fb6dad00996cee7d5342490daa7bb1fb
Date: 2024-05-31T01:18:07.094Z
Electron: 29.4.0
ElectronBuildId: 9593362
Chromium: 122.0.6261.156
Node.js: 20.9.0
V8: 12.2.281.27-electron.0
OS: Darwin arm64 23.5.0

I don't have any exclusion rules at the repo or org level. My account is eligible for the free "Copilot Individual subscription" b/c of opensource contributions though. After scanning through tickets here and the copilot community discussions, there are some mentions of similar issues, like the one that was caused by not having any changes staged (I don't have the issue number but it might be what @canuckjacq was referring to above).

I mention the free subscription b/c I'm just wondering if there's some kind of logic that's preventing accounts with acknowledged opensource contributions from auto-generating commit messages. For example as a way to avoid poisoning the well of organic, human written content. I don't know. It's a shot in the dark but it's been a real stick my craw so I've spent a lot of cycles thinking about it.

delano avatar Jun 02 '24 20:06 delano

Same issue here

visionplanner-admin avatar Jun 07 '24 12:06 visionplanner-admin

The latest VS Code Insiders + Copilot Chat pre-release contains some additional logging that should hopefully help us track down the root cause of the problem. Could you please install the latest VS Code Insiders + Copilot Chat pre-release extension, try to generate a commit message and share the logs as described here: https://github.com/microsoft/vscode-copilot-release/issues/1229#issuecomment-2135210792

lszomoru avatar Jun 07 '24 12:06 lszomoru

The latest VS Code Insiders + Copilot Chat pre-release contains some additional logging that should hopefully help us track down the root cause of the problem. Could you please install the latest VS Code Insiders + Copilot Chat pre-release extension, try to generate a commit message and share the logs as described here: #1229 (comment)

The following is the log I saw when opening VS Code Insiders, and no new logs were generated when I used "Generate Commit Message with Copilot" later.

2024-06-08 01:18:07.376 [info] [gitExtensionService] Initializing Git extension service.
2024-06-08 01:18:07.376 [info] [gitExtensionService] Successfully activated the vscode.git extension.
2024-06-08 01:18:07.376 [info] [gitExtensionService] Enablement state of the vscode.git extension: true.
2024-06-08 01:18:07.376 [info] [gitExtensionService] Successfully registered Git commit message provider.
2024-06-08 01:18:07.376 [info] [FetcherService] Using the Helix fetcher.
2024-06-08 01:18:07.409 [info] [auth] Logged in as liby
2024-06-08 01:18:08.189 [info] [chat] copilot token chat_enabled: true
2024-06-08 01:18:08.189 [info] [githubTitleAndDescriptionProvider] Initializing GitHub PR title and description provider provider.
2024-06-08 01:18:08.189 [info] [githubTitleAndDescriptionProvider] GitHub.vscode-pull-request-github extension is not yet activated.
2024-06-08 01:18:08.197 [info] [auth] Got Copilot token for liby
2024-06-08 01:18:09.552 [info] [RemoteAgents] Invalid remote agent response: 404 page not found
 (SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5))
2024-06-08 01:18:31.082 [info] [gitExtensionService] Initializing Git extension service.
2024-06-08 01:18:31.082 [info] [FetcherService] Using the Helix fetcher.
2024-06-08 01:18:31.082 [info] [auth] Logged in as liby
2024-06-08 01:18:31.082 [info] [gitExtensionService] Successfully activated the vscode.git extension.
2024-06-08 01:18:31.082 [info] [gitExtensionService] Enablement state of the vscode.git extension: true.
2024-06-08 01:18:31.082 [info] [gitExtensionService] Successfully registered Git commit message provider.
2024-06-08 01:18:31.799 [info] [chat] copilot token chat_enabled: true
2024-06-08 01:18:31.799 [info] [githubTitleAndDescriptionProvider] Initializing GitHub PR title and description provider provider.
2024-06-08 01:18:31.799 [info] [githubTitleAndDescriptionProvider] GitHub.vscode-pull-request-github extension is not yet activated.
2024-06-08 01:18:31.809 [info] [auth] Got Copilot token for liby
2024-06-08 01:18:32.948 [info] [RemoteAgents] Invalid remote agent response: 404 page not found
 (SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5))

liby avatar Jun 07 '24 17:06 liby

I see the same agent response error in my logs:

2024-06-08 01:18:32.948 [info] [RemoteAgents] Invalid remote agent response: 404 page not found
 (SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5))

I captured the HTTP traffic with mitmproxy and believe this is the request (reproduced as a curl command):

$ curl -v \
     -H 'authorization: Bearer REDACTED' \
     -H 'editor-version: vscode/1.91.0-insider' \
     -H 'editor-plugin-version: copilot-chat/0.17.2024060701' \
     -H 'user-agent: GitHubCopilotChat/0.17.2024060701' \
     -H 'accept: */*' --compressed https://api.githubcopilot.com/agents


< HTTP/2 404 
< content-security-policy: default-src 'none'; sandbox
< content-type: text/plain; charset=utf-8
< strict-transport-security: max-age=31536000
< x-content-type-options: nosniff
< date: Fri, 07 Jun 2024 18:16:53 GMT
< content-length: 19
< x-github-backend: Kubernetes
< x-github-request-id: D20A:16B6FF:10A1CD0:1601CED:66634E95

afb-requests-request-form-autosuggest-2024-03-25-5

Version: 1.91.0-insider
Commit: 6facfe23b300eb448c35d07291cad7ad98625302
Date: 2024-06-07T08:38:32.379Z
Electron: 29.4.0
ElectronBuildId: 9593362
Chromium: 122.0.6261.156
Node.js: 20.9.0
V8: 12.2.281.27-electron.0
OS: Darwin arm64 23.5.0

Output - GitHub Copilot Chat

2024-06-07 11:02:14.176 [info] [gitExtensionService] Initializing Git extension service.
2024-06-07 11:02:14.177 [info] [gitExtensionService] Successfully activated the vscode.git extension.
2024-06-07 11:02:14.177 [info] [gitExtensionService] Enablement state of the vscode.git extension: true.
2024-06-07 11:02:14.177 [info] [gitExtensionService] Successfully registered Git commit message provider.
2024-06-07 11:02:14.177 [info] [FetcherService] Using the Helix fetcher.
2024-06-07 11:02:14.246 [info] [auth] Logged in as delano
2024-06-07 11:02:14.713 [info] [chat] copilot token chat_enabled: true
2024-06-07 11:02:14.713 [info] [githubTitleAndDescriptionProvider] Initializing GitHub PR title and description provider provider.
2024-06-07 11:02:14.713 [info] [githubTitleAndDescriptionProvider] GitHub.vscode-pull-request-github extension is not yet activated.
2024-06-07 11:02:14.725 [info] [auth] Got Copilot token for delano
2024-06-07 11:02:15.395 [info] [RemoteAgents] Invalid remote agent response: 404 page not found
 (SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5))
2024-06-07 11:02:56.670 [info] [gitExtensionService] Initializing Git extension service.
2024-06-07 11:02:56.670 [info] [gitExtensionService] Successfully activated the vscode.git extension.
2024-06-07 11:02:56.670 [info] [gitExtensionService] Enablement state of the vscode.git extension: true.
2024-06-07 11:02:56.670 [info] [gitExtensionService] Successfully registered Git commit message provider.
2024-06-07 11:02:56.670 [info] [FetcherService] Using the Helix fetcher.
2024-06-07 11:02:56.707 [info] [auth] Logged in as delano
2024-06-07 11:02:57.216 [info] [chat] copilot token chat_enabled: true
2024-06-07 11:02:57.217 [info] [githubTitleAndDescriptionProvider] Initializing GitHub PR title and description provider provider.
2024-06-07 11:02:57.217 [info] [githubTitleAndDescriptionProvider] GitHub.vscode-pull-request-github extension is not yet activated.
2024-06-07 11:02:57.223 [info] [auth] Got Copilot token for delano
2024-06-07 11:02:57.732 [info] [RemoteAgents] Invalid remote agent response: 404 page not found
 (SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5))
Edit: Replaced the gif

This is the original where I didn't stage the changed before trying to generate the commit message. The behaviour is the same either way, I just wanted to remove the ambiguity.

afb-requests-request-form-autosuggest-2024-03-25-5

delano avatar Jun 07 '24 18:06 delano

What is the version of the GitHub Copilot Chat extension? Could you please double check to make sure that you are running v0.17.2024060701 (pre-release) or later? You should see logs similar to this, and logs for each resource that is skipped due to content exclusion:

2024-06-09 14:20:28.410 [info] [extension] [GitCommitMessageService] Changes (before context exclusion): 1 file(s)
2024-06-09 14:20:28.414 [info] [extension] [GitCommitMessageService] Changes (after context exclusion): 1 file(s)

lszomoru avatar Jun 09 '24 12:06 lszomoru

What is the version of the GitHub Copilot Chat extension? Could you please double check to make sure that you are running v0.17.2024060701 (pre-release) or later? You should see logs similar to this, and logs for each resource that is skipped due to content exclusion:

2024-06-09 14:20:28.410 [info] [extension] [GitCommitMessageService] Changes (before context exclusion): 1 file(s)
2024-06-09 14:20:28.414 [info] [extension] [GitCommitMessageService] Changes (after context exclusion): 1 file(s)

I am using this version, but I did not see similar logs appear. image

liby avatar Jun 10 '24 07:06 liby

I was running Github Copilot Chat v0.17.2024060701 (pre-release) as well (this screenshot is from Friday). And also didn't see those additional log lines.

Screenshot 2024-06-07 at 11 04 32

delano avatar Jun 10 '24 18:06 delano

I noticed that it 100% fails to generate commit messages for entirely new files

joeybab3 avatar Jun 10 '24 21:06 joeybab3

The issue started with GitHub Copilot Chat v16.0. The only option is to downgrade to v0.15.2 until the developers fix the problem.

igoros777 avatar Jun 11 '24 21:06 igoros777

Hmm, the earliest version of "GitHub Copilot Chat" I'm able to install is 0.16.0. I tried installing similarly aged "GitHub Copilot" as well but no dice, same error: "Chat failed to load. Please ensure that the GitHub Copilot Chat extension is up to date.". Maybe there's a minimum viable VS Code version?

Screenshot 2024-06-12 at 16 14 30

delano avatar Jun 12 '24 23:06 delano

Even if it is able to generate a commit message, the commit message makes no sense at all, brings in unrelated stuff and is just overal really dumb.

DennisJansenDev avatar Jun 14 '24 07:06 DennisJansenDev

The latest Copilot Chat pre-release extension (v0.17.2024061702 (pre-release)) contains some changes to logging so I was wondering if you can install it, attempt to generate a new commit message and share the logs from the "GitHub Copilot Chat" output channel. Thanks!

lszomoru avatar Jun 18 '24 08:06 lszomoru

I have only this when I click the generate message button, latest insiders and pre-release version of extension:

2024-06-19 13:12:56.053 [info] [GitCommitMessageService] Changes (before context exclusion): 1 file(s)
2024-06-19 13:12:56.053 [info] [GitCommitMessageService] Ignoring change due to missing originalUri and modifiedUri.
2024-06-19 13:12:56.053 [info] [GitCommitMessageService] Changes (after context exclusion): 0 file(s)

filipac avatar Jun 19 '24 10:06 filipac

@filipac, awesome, thank you! We are getting there. Based on the logs it seems that there is a bug in parsing the git diff output and extracting the resource Uri(s). Could you please open the integrated terminal, and run the following commands and share the output if possible:

  • git diff --cached
  • git diff

lszomoru avatar Jun 19 '24 10:06 lszomoru

When I click the generate message button, I see a similar log

2024-06-19 23:58:42.946 [info] [GitCommitMessageService] Changes (before context exclusion): 1 file(s)
2024-06-19 23:58:42.946 [info] [GitCommitMessageService] Ignoring change due to missing originalUri and modifiedUri.
2024-06-19 23:58:42.946 [info] [GitCommitMessageService] Changes (after context exclusion): 0 file(s)
image

The output of the git diff command is as follows: image

liby avatar Jun 19 '24 16:06 liby

Same for me.

2024-06-19 10:42:17.326 [info] [GitCommitMessageService] Changes (before context exclusion): 1 file(s)
2024-06-19 10:42:17.326 [info] [GitCommitMessageService] Ignoring change due to missing originalUri and modifiedUri.
2024-06-19 10:42:17.326 [info] [GitCommitMessageService] Changes (after context exclusion): 0 file(s)

vscode-copilot-commit-message-20240619

delano avatar Jun 19 '24 17:06 delano

@liby, @delano, thanks for sharing the information. Could you run the git diff commands from git bash so that I see the raw text as both of you seem to be having some special rendering of the git diff output. Thank you!

lszomoru avatar Jun 21 '24 12:06 lszomoru

@liby, @delano, thanks for sharing the information. Could you run the git diff commands from git bash so that I see the raw text as both of you seem to be having some special rendering of the git diff output. Thank you!

I think I've located the root cause. The diff tool I am currently using is Difftastic, not the built-in diff in Git.

My .gitconfig is here. The relevant configuration items are:

[diff]
    tool = difftastic

[difftool]
    prompt = false

[difftool "difftastic"]
    cmd = difft "$LOCAL" "$REMOTE"

[pager]
    difftool = true

I found that when I switched to Git's built-in diff, the automatic generate commit messages worked fine.

But I have been using Difftastic instead of Git's built-in diff for about 2 years. Before I discovered this issue, it had been working well with automatic generate commit messages feature, and I don't know what happened during that time.

For the same code changes, this is the output of Git's built-in diff:

diff --git a/scripts/git-sync.sh b/scripts/git-sync.sh
index 51f78f8..8bbdc0b 100755
--- a/scripts/git-sync.sh
+++ b/scripts/git-sync.sh
@@ -14,5 +14,5 @@ elif [ -n "$(git status)" ]; then
   git remote -v | grep fetch | awk '{print $2}' | git pull --rebase origin master
   git push
 else
-  echo ${MESSAGE}
+  echo "${MESSAGE}"
 fi

This is the output of command Difftastic:

scripts/git-sync.sh --- Bash
14   git remote -v | grep fetch | awk '{pr 14   git remote -v | grep fetch | awk '{pr
.. int $2}' | git pull --rebase origin mas .. int $2}' | git pull --rebase origin mas
.. ter                                     .. ter
15   git push                              15   git push
16 else                                    16 else
17   echo ${MESSAGE}                       17   echo "${MESSAGE}"
18 fi                                      18 fi

liby avatar Jun 21 '24 12:06 liby