cody icon indicating copy to clipboard operation
cody copied to clipboard

Edit (Fix): Provide tailored prompt information for the most popular diagnostic fixes

Open umpox opened this issue 1 year ago • 0 comments

Description

There are an assortment of highly popular diagnostics that "Ask Cody to Fix" will run on, see this dashboard: https://sourcegraph.looker.com/looks/1914

We can use this data to improve the success rate for "Ask Cody to Fix"

Steps:

  1. Update this dashboard to track persistence rate per diagnostic code + source. This should give us a good indication of the most successful or error-prone diagnostics that Cody attempts to fix. Also track initial persistence as an additional data point
  2. Using the data from point 1, prioritise highly popular diagnostics that have low persistence rates. We should look into if we can add tailored information or examples to the prompt.
  3. Measure the impact of point 2 on persistence rates for specific diagnostics

umpox avatar Apr 08 '24 11:04 umpox