honojs
AI-Powered Automatic Issue Triage
Summary
Implement an AI system to automatically reproduce and classify GitHub issues, reducing maintainer workload and improving response times.
Motivation
- Many issues are "not a bug" but configuration/usage problems
- Maintainers spend significant time on initial triage
- Real bugs get buried in noise
- Current manual process doesn't scale with project growth
Proposed Solution
AI bot that:
- Parses issue descriptions - Extract reproduction steps, environment details, expected vs actual behavior
- Generates reproduction test code - Create minimal test cases based on issue description
- Runs tests in sandbox environment - Execute tests across multiple runtimes (Node.js, Deno, Bun, Cloudflare Workers)
- Auto-classifies issues - Categorize as: Bug/Not-a-bug/Need-info/Documentation
Benefits
- 80% reduction in manual triage time
- 24h initial response time for all issues
- Better focus on real bugs for maintainers
- Improved documentation derived from common mistakes
- Consistent quality of initial responses
Technical Approach
- GitHub Actions for automation
- Claude Code for AI processing and test generation
- Sandbox environment for safe test execution
- Multi-runtime testing for comprehensive validation
Community Impact
This could become a template for other OSS projects facing similar triage challenges.
Would love to get community feedback on this idea! 🚀
🤖 Generated with Claude Code
I am interested in this implementation and will make a PoC. (I may not finish it forever and I don't mind at all what others do)
@yusukebe
- do you have any plans for an LLM model to use?
Providers such as OpenAI and Google offer free tier in exchange for learning use of the data. If you don't mind, I think these are the best to use.
- how much can you afford to pay for API usage per issue/per day?
- using Github Actions is fine, but Cloudflare Workers (cron) would be a good choice for these reasons. What do you think?
- Time-limited Context Caching can be used by batching several issues at once
- Easy to develop locally
Hi @goisaki
Thank you for the comment. Honestly, I don't have so many ideas now. Regarding 3, it's the best use of GitHub Actions since we want everything on GitHub to be as good as possible. If you create something like a proposal, I'll check it!
(I've finished my personal work and can finally get started on this task. Sorry for the wait.) I think that's a very good approach. Dosu seems to have a preview mode. Please give it a try.
2. Deployment Mode
You’ll be prompted to select a deployment mode for the channel. Your options are:
- Standard Setup: This gives you full control over the channels’s configuration and allows you to customize the channel to your needs.
- Preview Mode: This is the quickest way to test Dosu out without any user-facing impact. It will create a channel that with all user-facing features disabled, but will generate new preview response for you to review within the app.
I think it would be good to introduce it to Discussions on a trial basis.
In addition, we could adopt the following measures to reduce our response time on issues:
- Formalize enhancement-request requirements
- Require approval from a core contributor or a certain number of 👍 reactions before we tackle an enhancement.
- Systematize repro-code submissions for bug reports
- Grant the reporter write access to a
hono-community/repro-for-<issue-number>repo when the issue is filed. - If no reproduction code is submitted within a preset timeframe, mark the issue as “not planned.”
- Grant the reporter write access to a
@goisaki
I want to have another one. The search system for the user to find the usage of Hono from the official website, GitHub issues, and Discuss in the repos in honojs org by AI. There are many materials in the repos to help them, but these are not documented, and it's hard to do that manually.
@goisaki
Anyway, please ping me if you made any demos or proposals. I also try it if I have time. Thanks.