backseat.nvim icon indicating copy to clipboard operation
backseat.nvim copied to clipboard

Better fewshot.lua training data

Open james1236 opened this issue 1 year ago • 0 comments

I'm looking to improve the suggestions given by backseat. The current training data consists of 2 sample code snippets with the AI's intended response (the 4 messages in lua/fewshot.lua). Main issues are:

  • Pointless changes, like changing camelCase to snake_case for no apparent reason
  • Recommending pointless comments that just restate what's already there

With good code, the AI likes to make pointless suggestions almost as if it were filling a quota. The range of suggestions could be greater too, I especially like these:

  • The complex calculations could be extracted into a function with a descriptive name, e.g...
  • Deep nested conditionals affect readability. Prefer early returns or helper functions to simplify control flow and improve readability

The current training data is less than 1,000 tokens, and since each additional token of training data increases the cost for every individual request (since training data is sent with every request, no fine tuning available for gpt-3.5-turbo or 4 yet), it's best to keep the training data short.

If anyone would like to try and figure out some better training prompts, playground is a good place to do it. It can be a lot of fun and I welcome any discussion, thank you!

james1236 avatar Apr 04 '23 22:04 james1236