ecosystem
ecosystem copied to clipboard
dart-lang
General feedback on the issue triage bot
This issue is for collecting feedback on the issue triage bot; areas where it could improve, individual issues that it didn't triage well, possible prompt improvements, ...
Please feel free to create new comments here for feedback.
Issue/comment: https://github.com/dart-lang/sdk/issues/56002#issuecomment-2166897601
Feedback: The bot triaged this macro related issue into area-test, likely because the issue discussed failing tests. As this is a common scenario, area-test should be weighted down somehow or have even more explicit prompting around preferring functional area.
https://github.com/dart-lang/sdk/issues/56028#issuecomment-2172556309 would benefit from getting more context than the first comment of the issue.
Similarly to the previous comment, for transferred issues (like https://github.com/dart-lang/sdk/issues/56072), we should at least feed in the first and last comment if not all the comments.
We may also consider introducing a slight delay in triaging issues (2 min?). This would allow some time for manual triage.
We should auto-close Analyzer Feedback from IntelliJ issues; these almost universally do not contain information (are not customized over the default template that just sends version info).
See https://github.com/dart-lang/ecosystem/pull/274 for a few of the discussed improvements.
When an issue comment contains an image, we should pass that image into the gemini API (along with the issue comment's text).
Maybe pass the current docs to solve cases like: https://github.com/dart-lang/sdk/issues/56180? I'm not really sure what to suggest but the summary is off.
Yeah, I agree that that summary is off; you were asking for an analyzer code assist and it summarized it as a language enhancement request. I don't know if that means we need better prompting here, and the model is already well-trained on the Dart language. Perhaps a better model? General incremental improvements in gemini's capabilities? We're using gemini-1.5-flash to classify now, I could try a dry run w/ just regular gemini 1.5.
Maybe a way to ask the bot for a review? Than you could have a little more context into the new review
Maybe a way to ask the bot for a review? Than you could have a little more context into the new review
Do you mean to interact with the bot a bit to improve the summary? I think it would be reasonable to re-prompt it based on responses to its summary / classification comment.
Exactly. Maybe a keyword (or even pinging it) to ask for it to review the OP and the keyword comment. I believe if that is to become a thing, it should probably always appear at the end of its comments, like: "If you disagree with me, do this to make it so that I review this issue again".
Not sure what's going on here but I have the Dart SDK as a fork (intend on helping out soon). Both:
- https://github.com/dart-lang/sdk/issues/56597
- https://github.com/dart-lang/sdk/issues/56602
Have sent me an email with failed jobs and have not been able to place the tags correctly, only commenting.
Maybe there is a bug here? Not sure if this is the right place or if this should have been a new issue.
About the above comment, not sure if it has been addressed or what (maybe a tag-specific problem), but I just created a new issue https://github.com/dart-lang/sdk/issues/56631. Got no email errors and the bot could place the tags normally.
We had a regression in the triage bot - it was briefly trying to assign too many labels and the label assignment part of the workflow was failing; I suspect that's what you were seeing.
I see.
But why does the fork gets the email? I see that I have not received any other issue so only issues I wrote but still don't understand why am I receiving the emails? Is this a bug on GitHub Actions or something?
But why does the fork gets the email? I see that I have not received any other issue so only issues I wrote
Good question! I'm not sure why you're being notified. Perhaps it's related to your fork or perhaps related to you being the author of the issue that's triggering the workflow.
Had an issue with a type-question label, but no area- label, where the bot added an area (correctly) and a second type- label.
It should proabably not add any label for which there is already a label with the same prefix.
For this debugger related issue: https://github.com/dart-lang/sdk/issues/57040, it was triaged to area-language but should have been triaged to area-vm.
Issue https://github.com/dart-lang/sdk/issues/57055 was triaged into area-test; it should have been triaged into area-vm.
From https://github.com/dart-lang/sdk/issues/57064, it's difficult to know the right area to assign macros related issues.
I can generally say that area-language is more wrong than right. (There isn't much use for area-language in the SDK repo after the language repo was created, so it's unlikely that area-language will be correct.)