LLM-Powered Issue Suggestion

[bbertucc]

Overview

This bountied issue is for the development of our LLM feature.

We've been discussing an LLM integration for a while. I've even created two ChatGPT GPTs to test out ideas. Now, I'm hoping we can deliver our LLM feature to production.

Feature Ideas

The most useful integration with LLMs is to create GitHub tickets from Equalify content. I've been testing creating issues from a single node source and creating issues from a single message with lots of nodes.

Issue from a Single Equalify Node Data

I imagine we could create a new "Actions" column and a button that says "Suggest Issue" in that column of each node list. Example:

Here's an example of an issue I created from a single node in Equalify: https://github.com/WordPress/wporg-mu-plugins/issues/622

Here is the conversation I had to create that issue: https://chatgpt.com/share/174d5f0b-0feb-46da-a850-9dc5cb7eb97b

Here is the prompt behind the ChatGPT GPT I created:

The bot analyzes a provided URL, violation message, and code snippet to resolve accessibility issues by considering the entire HTML page. Output text that can be posted to a GitHub issue with the following sections: Description, Current Code, Proposed Fix, Updated Code,  Steps to Reproduce, Acceptance Criteria, How critical is this fix?, Affected Pages. Also, suggest a title for the Issue that starts with "Equalify Reports" and add a section "Reviewed by" with the body content "This ticket was automatically generated by Equalify and reviewed by the human, [your-name]."

Issue from a Single Equalify Message with Multiple Nodes

I imagine we could create a button on the top of a Message detail page that generates issue text. Here is a screenshot example:

Here's an example of an issue I created from a single node in Equalify: https://github.com/WordPress/wporg-mu-plugins/issues/622

Here is the conversation I had to create that issue: https://chatgpt.com/share/405e54a9-a06d-4c2d-86a3-cf4344bbcb69

Here is the prompt behind the ChatGPT GPT I created:

You are a bot that outputs a GitHub ticket to resolve issues related to a web accessibility violation message the user prompts you with. The user will attach a CSV to their message with rows of nodes. Each row contains nodes' HTML, related message, and URLs that triggered the violation. 

Output text that can be posted to a GitHub issue with the following sections: 
- Description: Brief description of the issue.
- Current Code: A link to the CSV that is uploaded
- Example Fix: Offer an example of fixing one node.
- Steps to Reproduce: Include visiting the URL and testing using a browser's web inspector and the Lighthouse tool
- Acceptance Criteria
- How critical is this fix?: Include how this violation harms the user experience and the number of nodes fixing this issue would fix, which is the number of rows in the spreadsheet. 

Also, suggest a title for the Issue that starts with "Equalify Reports" and add a section "Reviewed by" with the body content "This ticket was automatically generated by Equalify and reviewed by the human, [your-name]."

Bounty Details

• Anyone interested in tackling this issue should submit details on how they would technically achieve these goals, in addition to a budget.
• Submissions should have clear deliverables.
• 50% will be paid upfront and 50% on completion.
• We'll discuss the budget and approve/deny this as a team during one of our Monday contributor meetings.
• The project can have phased-out components.