[Bug][Github] Table 'pull_request_assignees' & 'pull_request_reviewers' has no user data from Github

[lokeshmandhare]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

Under Github dashboard thats provided in-built with Devlake Grafana, we have a requirement of creating panels which have PR assignee and PR reviewer data. However to create these panels, the reference tables 'pull_request_assignees' & 'pull_request_reviewers' has no user data in Devlake's MySQL DB. Can anybody check why this is and does Devlake fail to pull this data from Github ?

What do you expect to happen

Assignee and Reviewer data should reflect in those respective tables in Devlake's MySQL DB so it can be used to create panel in Grafana

How to reproduce

Login to Devlake MySQL DB Look for tables 'pull_request_assignees' & 'pull_request_reviewers'

Anything else

No response

Version

v1.0.0@44f3dea

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[dosubot[bot]]

Hello @lokeshmandhare! I'm here to help you with any bugs, questions, or contributions you have for the repository. Let's work together to resolve the issue you're facing.

The issue you're experiencing with the 'pull_request_assignees' and 'pull_request_reviewers' tables not containing user data from GitHub in DevLake's MySQL database is likely due to the migration script modifyPrAssigneeAndReviewerId. This script deletes the existing tables and their migration history, and then recreates them, which results in the loss of any previously stored data in these tables ^[1].

To ensure that user data is correctly ingested into these tables, you need to follow the specific steps and configurations for setting up the GitHub integration in DevLake:

1. Add Data Connections:

   • Connection Name: Unique identifier for the connection.
   • Endpoint URL: Use https://api.github.com/.
   • Personal Access Token(s): Ensure the token has the required permissions:
     • For public repositories: repo:status, repo_deployment, read:user, read:org.
     • For private repositories: repo, read:user, read:org.
   • Use GraphQL APIs: Enable if supported.
   • Test and Save Connection.

2. Add Data Scopes:

   • Select the GitHub repositories for data collection.

3. Add Scope Config (Optional):

   • Configure the Code Review section to parse PR labels.

4. Create a Project:

   • Create a new project in the Project page.

5. Add a GitHub Connection to the Project:

   • Add the configured GitHub connection and select repositories.

6. Set the Sync Policy:

   • Data Time Range: Select the desired time range.
   • Sync Frequency: Choose the sync frequency.
   • Skip Failed Tasks: Optionally enable this.

7. Start Data Collection:

   • Click "Collect Data" to start the process.

Following these steps should ensure that DevLake correctly ingests user data into the pull_request_assignees and pull_request_reviewers tables ^[2][3][4].

Additionally, there is a known issue in version v1.0.0 related to the GitExtractor task failing in GitHub blueprints, which could impact data collection. This issue is documented and was fixed in a later version ^[5].

To continue talking to Dosu, mention @dosu.

[klesh]

Gitub plugin doesn't collect these information at the moment.

[klesh]

@Startrekzky

[d4x1]

@lokeshmandhare We've tested the latest codes in main branch. Table pull_request_reviewers have expected records. You can try latest version: https://github.com/apache/incubator-devlake/releases/tag/v1.0.1-beta6. And in 44f3dea, it's normal that table pull_request_reviewers is empty.

As far table pull_request_assignees , it's empty in main branch ans that's as expected.

[lokeshmandhare]

So @d4x1 , should I try the steps suggested by dosubot or are you recommending me to upgrade to the latest version to fix this. And regarding pull_request_assignees, are you saying that even if the Github PR has a assignee to it, it still wont be pulled into the Devlakes MySQL DB ? If thats the case, what are the steps to create a panel with 'PR Assignee' info ?

[d4x1]

@lokeshmandhare pull_request_assignees is empty means even there is a table pull_request_assignees in DevLake, but there is no code that write pull_requests' assginees to this table in plugin github, so it's empty. DevLake has no dashboard requires such data now. (If you're interested in this work, any PRs are welcomed.)

And there are some code to write to table pull_request_reviewers in version: https://github.com/apache/incubator-devlake/releases/tag/v1.0.1-beta6.

You can try the latest version.

[lokeshmandhare]

Thanks @d4x1, I will try the latest version for pulling 'pull_request_reviewers' and update if it worked.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.