[label bot] Use text in additional comments not just the initial comment

[jlewi]

We should use the text from all comments not just the first comment to try to predict issue labels.

Hopefully subsequent discussion in the issue will help clarify what the issue is about and lead to better label predictions.

There are multiple pieces involved in this

• We need to modify the front end to send events to the workers for comment events not just issue created events
• the workers need to fetch all comments not just the first one
• the workers need to look at what labels have already been applied and if any labels have been explicitly removed and not include them in the predictions.
• For training we should experiment with using all the comments
  • We might also want to consider treating each (comment, issue_label) as a training example.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the labels:

Label	Probability
kind/feature	0.73

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback! Links: app homepage, dashboard and code for this bot.

[jlewi]

• Draft PRs are open with both backend (#138) and front end changes (machine-learning-apps/Issue-Label-Bot#72)

• The backend has been deployed to dev (but not prod)

• I'm currently deploying the frontend to dev

We should verify things are working in dev before promoting to prod.

I did some verification of the backend see test issue kubeflow/code-intelligence#137

The backend seems to be properly updating issues and taking into account existing labels correctly.

[jlewi]

I temporarily enabled the github app kf-label-bot-dev on kubeflow/kubeflow to test it out and likewise disabled issue-label-pod (the prod instance) on kubeflow/kubeflow.

[jlewi]

Some follow on work

• @hamelsmu suggested here adding some sort of feature flag to let the model learn that additional text is comments.

• AutoML doesn't give us a lot of flexibility to provide additional features; it just takes a text file containing the document.

• One thing we could do though is potentially use a model to predict label based on comment and then train some other model to combine the predictions from all the comments; some sort of mixture of experts.

[jlewi]

• The backend changes #138 have been pushed to prod.

[jlewi]

• The frontend changes have been rolled out to prod.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the labels:

Label	Probability
area/engprod	0.81

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback! Links: app homepage, dashboard and code for this bot.

[jlewi]

Cool! That comment indicates it is working.

[jlewi]

I turned off issue-label-bot-dev on kubeflow/kubeflow. Looks likes like label-bot was still enabled on kubeflow/kubeflow so I didn't need to reenable it.

[hamelsmu]

👍