app icon indicating copy to clipboard operation
app copied to clipboard

Feature Request: AI Assistant Context Retention for Test Failure Analysis & Error Clustering

Open seusur opened this issue 10 months ago • 0 comments

Description:

Currently, when using the "Explain Failure" feature in the AI Assistant, it provides an initial analysis but does not retain context from previous explanations. If a user provides additional details to improve the assistant’s understanding, it starts giving more accurate responses. However, once the chat window is closed and reopened, the assistant forgets previous explanations and gives an incorrect or incomplete analysis again.

Additionally, when using the "Clusterise Errors" feature in Run Report, the AI does not take into account past explanations or manually identified patterns, which reduces the accuracy of error grouping.

Image

Image

Image

Image

Expected Behavior:

  1. The AI Assistant should remember past explanations and context related to specific test failures.
  2. It should retain user-provided insights across sessions to improve future failure analysis.
  3. This will allow for more accurate and relevant explanations over time.

Proposed Solution:

  1. Implement persistent memory for the AI Assistant to store past user feedback and explanations.
  2. Link AI insights to automation test runs or recurring failure patterns.
  3. Provide an option for users to edit or refine stored explanations if needed.
  4. Ensure that past context is used in the "Clusterise Errors" function, allowing better grouping of similar failures based on prior learnings.

seusur avatar Mar 19 '25 12:03 seusur