CodeGPT icon indicating copy to clipboard operation
CodeGPT copied to clipboard

Commit message generation doesn't actually send git diff to LLM

Open Bikkies opened this issue 2 years ago • 5 comments

What happened?

When using the "Generate Message" button in the commit interface, only the prompt to generate the message is sent to the LLM, and none of the actual git diff. This means that the LLM outputs a completely made up list of changes.

Relevant log output or stack trace

13:10:58-614334 INFO     PROMPT=
<|begin▁of▁sentence|>Write a short and descriptive git commit message for the following git diff.
Use imperative mood, present tense, active voice and verbs.
Your entire response will be passed directly into git commit.### Response:


Output generated in 3.84 seconds (26.05 tokens/s, 100 tokens, context 62, seed 2126101025)

Steps to reproduce

Using PHPStorm 2023.3.5 and an openai-compatible model provider with logging that allows you to see the raw prompt to the LLM.

Make changes to the code and ensure there are changes in the git diff.

Open the commit interface and click the "Generate Message" button.

Observe that only the prompt to generate the message is sent to the LLM along with no git diff.

Observe that the response is completely made up as the LLM has no context of the actual git diff

CodeGPT version

2.5.1

Operating System

Windows

Bikkies avatar Mar 21 '24 03:03 Bikkies

I have the same problem

a8b7 avatar Mar 22 '24 02:03 a8b7

same here: https://github.com/carlrobertoh/CodeGPT/issues/390#issuecomment-2009208391

razisam avatar Mar 22 '24 12:03 razisam

I wanted to fix this, since i use this feature a lot, it seems it is already fixed in the current master, at least I was unable to reproduce it in the development IDE.

For my actual work I use PHPStorm, I'll try the build there.

rwese avatar Apr 04 '24 18:04 rwese

I will release the new stuff sometime early next week.

carlrobertoh avatar Apr 06 '24 11:04 carlrobertoh

@carlrobertoh update, the issue is reproducible, on main, using the gpt-4 turbo (128k) model.

If I find some time I'll try and look into it.

rwese avatar Apr 14 '24 16:04 rwese

imho: this is fixed aka "works for me" after the update

vroomfondel avatar May 21 '24 11:05 vroomfondel