oso icon indicating copy to clipboard operation
oso copied to clipboard

Refactor `stg_github__push_events`

Open ccerv1 opened this issue 2 months ago • 1 comments

ccerv1 avatar Nov 05 '25 17:11 ccerv1

OSO-1265

linear[bot] avatar Nov 05 '25 17:11 linear[bot]

Findings:

  1. The data are updated after 7 Oct officially. From what we received from gharchive, there seems to be a bit delay but we should assume the data since 7 Oct are altered.
  2. There are significant sudden drop on the amount of data around the time 7 Oct. Events for COMMIT_CODE, and PULL_REQUEST_* are dropped to 0 after the changes.
  3. There are no official documentation stated clearly on the changes of payload, but only a high level description announcement. (https://github.blog/changelog/2025-08-08-upcoming-changes-to-github-events-api-payloads/)
  4. This is not sufficient to list out all the changes, but it's a good starting point to refer to https://github.com/github/docs/commit/db8ac67d7424f61381d10e122702ffa794364a96

Payload Differences Raw Log

=== CommitCommentEvent ===
{'dictionary_item_added': ["root['action']"],
 'dictionary_item_removed': ["root['comment']['author_association']"]}

=== CreateEvent ===
{'dictionary_item_added': ["root['full_ref']"],
 'type_changes': {"root['ref']": {'new_type': <class 'str'>,
                                  'old_type': <class 'NoneType'>}}}

=== DeleteEvent ===
{'dictionary_item_added': ["root['full_ref']"]}

=== ForkEvent ===
{'dictionary_item_added': ["root['action']"],
 'dictionary_item_removed': ["root['forkee']['public']"],
 'type_changes': {"root['forkee']['license']": {'new_type': <class 'dict'>,
                                                'old_type': <class 'NoneType'>}}}

=== GollumEvent ===
{}

=== IssueCommentEvent ===
{'dictionary_item_added': ["root['issue']['draft']", "root['issue']['pull_request']"],
 'dictionary_item_removed': ["root['issue']['author_association']", "root['issue']['sub_issues_summary']", "root['comment']['author_association']"],
 'iterable_item_removed': {"root['issue']['labels'][2]": {'color': '106A2F',
                                                          'default': False,
                                                          'description': '',
                                                          'id': 6414462636,
                                                          'name': 'confirmed',
                                                          'node_id': 'LA_kwDOA0-Arc8AAAABflTurA',
                                                          'url': 'https://api.github.com/repos/lord-server/lord/labels/confirmed'}},
 'type_changes': {"root['comment']['performed_via_github_app']": {'new_type': <class 'dict'>,
                                                                  'old_type': <class 'NoneType'>},
                  "root['issue']['closed_at']": {'new_type': <class 'str'>,
                                                 'old_type': <class 'NoneType'>}}}

=== IssuesEvent ===
{'dictionary_item_added': ["root['issue']['issue_dependencies_summary']"],
 'dictionary_item_removed': ["root['issue']['author_association']"],
 'type_changes': {"root['issue']['body']": {'new_type': <class 'NoneType'>,
                                            'old_type': <class 'str'>},
                  "root['issue']['closed_at']": {'new_type': <class 'NoneType'>,
                                                 'old_type': <class 'str'>},
                  "root['issue']['state_reason']": {'new_type': <class 'NoneType'>,
                                                    'old_type': <class 'str'>}}}

=== MemberEvent ===
{}

=== PublicEvent ===
{}

=== PullRequestEvent ===
{'dictionary_item_added': ["root['label']", "root['labels']"],
 'dictionary_item_removed': ["root['pull_request']['node_id']", "root['pull_request']['html_url']", "root['pull_request']['diff_url']", "root['pull_request']['patch_url']", "root['pull_request']['issue_url']", "root['pull_request']['state']", "root['pull_request']['locked']", "root['pull_request']['title']", "root['pull_request']['user']", "root['pull_request']['body']", "root['pull_request']['created_at']", "root['pull_request']['updated_at']", "root['pull_request']['closed_at']", "root['pull_request']['merged_at']", "root['pull_request']['merge_commit_sha']", "root['pull_request']['assignee']", "root['pull_request']['assignees']", "root['pull_request']['requested_reviewers']", "root['pull_request']['requested_teams']", "root['pull_request']['labels']", "root['pull_request']['milestone']", "root['pull_request']['draft']", "root['pull_request']['commits_url']", "root['pull_request']['review_comments_url']", "root['pull_request']['review_comment_url']", "root['pull_request']['comments_url']", "root['pull_request']['statuses_url']", "root['pull_request']['_links']", "root['pull_request']['author_association']", "root['pull_request']['auto_merge']", "root['pull_request']['active_lock_reason']", "root['pull_request']['merged']", "root['pull_request']['mergeable']", "root['pull_request']['rebaseable']", "root['pull_request']['mergeable_state']", "root['pull_request']['merged_by']", "root['pull_request']['comments']", "root['pull_request']['review_comments']", "root['pull_request']['maintainer_can_modify']", "root['pull_request']['commits']", "root['pull_request']['additions']", "root['pull_request']['deletions']", "root['pull_request']['changed_files']", "root['pull_request']['head']['label']", "root['pull_request']['head']['user']", "root['pull_request']['head']['repo']['node_id']", "root['pull_request']['head']['repo']['full_name']", "root['pull_request']['head']['repo']['private']", "root['pull_request']['head']['repo']['owner']", "root['pull_request']['head']['repo']['html_url']", "root['pull_request']['head']['repo']['description']", "root['pull_request']['head']['repo']['fork']", "root['pull_request']['head']['repo']['forks_url']", "root['pull_request']['head']['repo']['keys_url']", "root['pull_request']['head']['repo']['collaborators_url']", "root['pull_request']['head']['repo']['teams_url']", "root['pull_request']['head']['repo']['hooks_url']", "root['pull_request']['head']['repo']['issue_events_url']", "root['pull_request']['head']['repo']['events_url']", "root['pull_request']['head']['repo']['assignees_url']", "root['pull_request']['head']['repo']['branches_url']", "root['pull_request']['head']['repo']['tags_url']", "root['pull_request']['head']['repo']['blobs_url']", "root['pull_request']['head']['repo']['git_tags_url']", "root['pull_request']['head']['repo']['git_refs_url']", "root['pull_request']['head']['repo']['trees_url']", "root['pull_request']['head']['repo']['statuses_url']", "root['pull_request']['head']['repo']['languages_url']", "root['pull_request']['head']['repo']['stargazers_url']", "root['pull_request']['head']['repo']['contributors_url']", "root['pull_request']['head']['repo']['subscribers_url']", "root['pull_request']['head']['repo']['subscription_url']", "root['pull_request']['head']['repo']['commits_url']", "root['pull_request']['head']['repo']['git_commits_url']", "root['pull_request']['head']['repo']['comments_url']", "root['pull_request']['head']['repo']['issue_comment_url']", "root['pull_request']['head']['repo']['contents_url']", "root['pull_request']['head']['repo']['compare_url']", "root['pull_request']['head']['repo']['merges_url']", "root['pull_request']['head']['repo']['archive_url']", "root['pull_request']['head']['repo']['downloads_url']", "root['pull_request']['head']['repo']['issues_url']", "root['pull_request']['head']['repo']['pulls_url']", "root['pull_request']['head']['repo']['milestones_url']", "root['pull_request']['head']['repo']['notifications_url']", "root['pull_request']['head']['repo']['labels_url']", "root['pull_request']['head']['repo']['releases_url']", "root['pull_request']['head']['repo']['deployments_url']", "root['pull_request']['head']['repo']['created_at']", "root['pull_request']['head']['repo']['updated_at']", "root['pull_request']['head']['repo']['pushed_at']", "root['pull_request']['head']['repo']['git_url']", "root['pull_request']['head']['repo']['ssh_url']", "root['pull_request']['head']['repo']['clone_url']", "root['pull_request']['head']['repo']['svn_url']", "root['pull_request']['head']['repo']['homepage']", "root['pull_request']['head']['repo']['size']", "root['pull_request']['head']['repo']['stargazers_count']", "root['pull_request']['head']['repo']['watchers_count']", "root['pull_request']['head']['repo']['language']", "root['pull_request']['head']['repo']['has_issues']", "root['pull_request']['head']['repo']['has_projects']", "root['pull_request']['head']['repo']['has_downloads']", "root['pull_request']['head']['repo']['has_wiki']", "root['pull_request']['head']['repo']['has_pages']", "root['pull_request']['head']['repo']['has_discussions']", "root['pull_request']['head']['repo']['forks_count']", "root['pull_request']['head']['repo']['mirror_url']", "root['pull_request']['head']['repo']['archived']", "root['pull_request']['head']['repo']['disabled']", "root['pull_request']['head']['repo']['open_issues_count']", "root['pull_request']['head']['repo']['license']", "root['pull_request']['head']['repo']['allow_forking']", "root['pull_request']['head']['repo']['is_template']", "root['pull_request']['head']['repo']['web_commit_signoff_required']", "root['pull_request']['head']['repo']['topics']", "root['pull_request']['head']['repo']['visibility']", "root['pull_request']['head']['repo']['forks']", "root['pull_request']['head']['repo']['open_issues']", "root['pull_request']['head']['repo']['watchers']", "root['pull_request']['head']['repo']['default_branch']", "root['pull_request']['base']['label']", "root['pull_request']['base']['user']", "root['pull_request']['base']['repo']['node_id']", "root['pull_request']['base']['repo']['full_name']", "root['pull_request']['base']['repo']['private']", "root['pull_request']['base']['repo']['owner']", "root['pull_request']['base']['repo']['html_url']", "root['pull_request']['base']['repo']['description']", "root['pull_request']['base']['repo']['fork']", "root['pull_request']['base']['repo']['forks_url']", "root['pull_request']['base']['repo']['keys_url']", "root['pull_request']['base']['repo']['collaborators_url']", "root['pull_request']['base']['repo']['teams_url']", "root['pull_request']['base']['repo']['hooks_url']", "root['pull_request']['base']['repo']['issue_events_url']", "root['pull_request']['base']['repo']['events_url']", "root['pull_request']['base']['repo']['assignees_url']", "root['pull_request']['base']['repo']['branches_url']", "root['pull_request']['base']['repo']['tags_url']", "root['pull_request']['base']['repo']['blobs_url']", "root['pull_request']['base']['repo']['git_tags_url']", "root['pull_request']['base']['repo']['git_refs_url']", "root['pull_request']['base']['repo']['trees_url']", "root['pull_request']['base']['repo']['statuses_url']", "root['pull_request']['base']['repo']['languages_url']", "root['pull_request']['base']['repo']['stargazers_url']", "root['pull_request']['base']['repo']['contributors_url']", "root['pull_request']['base']['repo']['subscribers_url']", "root['pull_request']['base']['repo']['subscription_url']", "root['pull_request']['base']['repo']['commits_url']", "root['pull_request']['base']['repo']['git_commits_url']", "root['pull_request']['base']['repo']['comments_url']", "root['pull_request']['base']['repo']['issue_comment_url']", "root['pull_request']['base']['repo']['contents_url']", "root['pull_request']['base']['repo']['compare_url']", "root['pull_request']['base']['repo']['merges_url']", "root['pull_request']['base']['repo']['archive_url']", "root['pull_request']['base']['repo']['downloads_url']", "root['pull_request']['base']['repo']['issues_url']", "root['pull_request']['base']['repo']['pulls_url']", "root['pull_request']['base']['repo']['milestones_url']", "root['pull_request']['base']['repo']['notifications_url']", "root['pull_request']['base']['repo']['labels_url']", "root['pull_request']['base']['repo']['releases_url']", "root['pull_request']['base']['repo']['deployments_url']", "root['pull_request']['base']['repo']['created_at']", "root['pull_request']['base']['repo']['updated_at']", "root['pull_request']['base']['repo']['pushed_at']", "root['pull_request']['base']['repo']['git_url']", "root['pull_request']['base']['repo']['ssh_url']", "root['pull_request']['base']['repo']['clone_url']", "root['pull_request']['base']['repo']['svn_url']", "root['pull_request']['base']['repo']['homepage']", "root['pull_request']['base']['repo']['size']", "root['pull_request']['base']['repo']['stargazers_count']", "root['pull_request']['base']['repo']['watchers_count']", "root['pull_request']['base']['repo']['language']", "root['pull_request']['base']['repo']['has_issues']", "root['pull_request']['base']['repo']['has_projects']", "root['pull_request']['base']['repo']['has_downloads']", "root['pull_request']['base']['repo']['has_wiki']", "root['pull_request']['base']['repo']['has_pages']", "root['pull_request']['base']['repo']['has_discussions']", "root['pull_request']['base']['repo']['forks_count']", "root['pull_request']['base']['repo']['mirror_url']", "root['pull_request']['base']['repo']['archived']", "root['pull_request']['base']['repo']['disabled']", "root['pull_request']['base']['repo']['open_issues_count']", "root['pull_request']['base']['repo']['license']", "root['pull_request']['base']['repo']['allow_forking']", "root['pull_request']['base']['repo']['is_template']", "root['pull_request']['base']['repo']['web_commit_signoff_required']", "root['pull_request']['base']['repo']['topics']", "root['pull_request']['base']['repo']['visibility']", "root['pull_request']['base']['repo']['forks']", "root['pull_request']['base']['repo']['open_issues']", "root['pull_request']['base']['repo']['watchers']", "root['pull_request']['base']['repo']['default_branch']"]}

=== PullRequestReviewEvent ===
{'dictionary_item_added': ["root['review']['updated_at']"],
 'dictionary_item_removed': ["root['review']['author_association']", "root['pull_request']['node_id']", "root['pull_request']['html_url']", "root['pull_request']['diff_url']", "root['pull_request']['patch_url']", "root['pull_request']['issue_url']", "root['pull_request']['state']", "root['pull_request']['locked']", "root['pull_request']['title']", "root['pull_request']['user']", "root['pull_request']['body']", "root['pull_request']['created_at']", "root['pull_request']['updated_at']", "root['pull_request']['closed_at']", "root['pull_request']['merged_at']", "root['pull_request']['merge_commit_sha']", "root['pull_request']['assignee']", "root['pull_request']['assignees']", "root['pull_request']['requested_reviewers']", "root['pull_request']['requested_teams']", "root['pull_request']['labels']", "root['pull_request']['milestone']", "root['pull_request']['draft']", "root['pull_request']['commits_url']", "root['pull_request']['review_comments_url']", "root['pull_request']['review_comment_url']", "root['pull_request']['comments_url']", "root['pull_request']['statuses_url']", "root['pull_request']['_links']", "root['pull_request']['author_association']", "root['pull_request']['auto_merge']", "root['pull_request']['active_lock_reason']", "root['pull_request']['head']['label']", "root['pull_request']['head']['user']", "root['pull_request']['head']['repo']['node_id']", "root['pull_request']['head']['repo']['full_name']", "root['pull_request']['head']['repo']['private']", "root['pull_request']['head']['repo']['owner']", "root['pull_request']['head']['repo']['html_url']", "root['pull_request']['head']['repo']['description']", "root['pull_request']['head']['repo']['fork']", "root['pull_request']['head']['repo']['forks_url']", "root['pull_request']['head']['repo']['keys_url']", "root['pull_request']['head']['repo']['collaborators_url']", "root['pull_request']['head']['repo']['teams_url']", "root['pull_request']['head']['repo']['hooks_url']", "root['pull_request']['head']['repo']['issue_events_url']", "root['pull_request']['head']['repo']['events_url']", "root['pull_request']['head']['repo']['assignees_url']", "root['pull_request']['head']['repo']['branches_url']", "root['pull_request']['head']['repo']['tags_url']", "root['pull_request']['head']['repo']['blobs_url']", "root['pull_request']['head']['repo']['git_tags_url']", "root['pull_request']['head']['repo']['git_refs_url']", "root['pull_request']['head']['repo']['trees_url']", "root['pull_request']['head']['repo']['statuses_url']", "root['pull_request']['head']['repo']['languages_url']", "root['pull_request']['head']['repo']['stargazers_url']", "root['pull_request']['head']['repo']['contributors_url']", "root['pull_request']['head']['repo']['subscribers_url']", "root['pull_request']['head']['repo']['subscription_url']", "root['pull_request']['head']['repo']['commits_url']", "root['pull_request']['head']['repo']['git_commits_url']", "root['pull_request']['head']['repo']['comments_url']", "root['pull_request']['head']['repo']['issue_comment_url']", "root['pull_request']['head']['repo']['contents_url']", "root['pull_request']['head']['repo']['compare_url']", "root['pull_request']['head']['repo']['merges_url']", "root['pull_request']['head']['repo']['archive_url']", "root['pull_request']['head']['repo']['downloads_url']", "root['pull_request']['head']['repo']['issues_url']", "root['pull_request']['head']['repo']['pulls_url']", "root['pull_request']['head']['repo']['milestones_url']", "root['pull_request']['head']['repo']['notifications_url']", "root['pull_request']['head']['repo']['labels_url']", "root['pull_request']['head']['repo']['releases_url']", "root['pull_request']['head']['repo']['deployments_url']", "root['pull_request']['head']['repo']['created_at']", "root['pull_request']['head']['repo']['updated_at']", "root['pull_request']['head']['repo']['pushed_at']", "root['pull_request']['head']['repo']['git_url']", "root['pull_request']['head']['repo']['ssh_url']", "root['pull_request']['head']['repo']['clone_url']", "root['pull_request']['head']['repo']['svn_url']", "root['pull_request']['head']['repo']['homepage']", "root['pull_request']['head']['repo']['size']", "root['pull_request']['head']['repo']['stargazers_count']", "root['pull_request']['head']['repo']['watchers_count']", "root['pull_request']['head']['repo']['language']", "root['pull_request']['head']['repo']['has_issues']", "root['pull_request']['head']['repo']['has_projects']", "root['pull_request']['head']['repo']['has_downloads']", "root['pull_request']['head']['repo']['has_wiki']", "root['pull_request']['head']['repo']['has_pages']", "root['pull_request']['head']['repo']['has_discussions']", "root['pull_request']['head']['repo']['forks_count']", "root['pull_request']['head']['repo']['mirror_url']", "root['pull_request']['head']['repo']['archived']", "root['pull_request']['head']['repo']['disabled']", "root['pull_request']['head']['repo']['open_issues_count']", "root['pull_request']['head']['repo']['license']", "root['pull_request']['head']['repo']['allow_forking']", "root['pull_request']['head']['repo']['is_template']", "root['pull_request']['head']['repo']['web_commit_signoff_required']", "root['pull_request']['head']['repo']['topics']", "root['pull_request']['head']['repo']['visibility']", "root['pull_request']['head']['repo']['forks']", "root['pull_request']['head']['repo']['open_issues']", "root['pull_request']['head']['repo']['watchers']", "root['pull_request']['head']['repo']['default_branch']", "root['pull_request']['base']['label']", "root['pull_request']['base']['user']", "root['pull_request']['base']['repo']['node_id']", "root['pull_request']['base']['repo']['full_name']", "root['pull_request']['base']['repo']['private']", "root['pull_request']['base']['repo']['owner']", "root['pull_request']['base']['repo']['html_url']", "root['pull_request']['base']['repo']['description']", "root['pull_request']['base']['repo']['fork']", "root['pull_request']['base']['repo']['forks_url']", "root['pull_request']['base']['repo']['keys_url']", "root['pull_request']['base']['repo']['collaborators_url']", "root['pull_request']['base']['repo']['teams_url']", "root['pull_request']['base']['repo']['hooks_url']", "root['pull_request']['base']['repo']['issue_events_url']", "root['pull_request']['base']['repo']['events_url']", "root['pull_request']['base']['repo']['assignees_url']", "root['pull_request']['base']['repo']['branches_url']", "root['pull_request']['base']['repo']['tags_url']", "root['pull_request']['base']['repo']['blobs_url']", "root['pull_request']['base']['repo']['git_tags_url']", "root['pull_request']['base']['repo']['git_refs_url']", "root['pull_request']['base']['repo']['trees_url']", "root['pull_request']['base']['repo']['statuses_url']", "root['pull_request']['base']['repo']['languages_url']", "root['pull_request']['base']['repo']['stargazers_url']", "root['pull_request']['base']['repo']['contributors_url']", "root['pull_request']['base']['repo']['subscribers_url']", "root['pull_request']['base']['repo']['subscription_url']", "root['pull_request']['base']['repo']['commits_url']", "root['pull_request']['base']['repo']['git_commits_url']", "root['pull_request']['base']['repo']['comments_url']", "root['pull_request']['base']['repo']['issue_comment_url']", "root['pull_request']['base']['repo']['contents_url']", "root['pull_request']['base']['repo']['compare_url']", "root['pull_request']['base']['repo']['merges_url']", "root['pull_request']['base']['repo']['archive_url']", "root['pull_request']['base']['repo']['downloads_url']", "root['pull_request']['base']['repo']['issues_url']", "root['pull_request']['base']['repo']['pulls_url']", "root['pull_request']['base']['repo']['milestones_url']", "root['pull_request']['base']['repo']['notifications_url']", "root['pull_request']['base']['repo']['labels_url']", "root['pull_request']['base']['repo']['releases_url']", "root['pull_request']['base']['repo']['deployments_url']", "root['pull_request']['base']['repo']['created_at']", "root['pull_request']['base']['repo']['updated_at']", "root['pull_request']['base']['repo']['pushed_at']", "root['pull_request']['base']['repo']['git_url']", "root['pull_request']['base']['repo']['ssh_url']", "root['pull_request']['base']['repo']['clone_url']", "root['pull_request']['base']['repo']['svn_url']", "root['pull_request']['base']['repo']['homepage']", "root['pull_request']['base']['repo']['size']", "root['pull_request']['base']['repo']['stargazers_count']", "root['pull_request']['base']['repo']['watchers_count']", "root['pull_request']['base']['repo']['language']", "root['pull_request']['base']['repo']['has_issues']", "root['pull_request']['base']['repo']['has_projects']", "root['pull_request']['base']['repo']['has_downloads']", "root['pull_request']['base']['repo']['has_wiki']", "root['pull_request']['base']['repo']['has_pages']", "root['pull_request']['base']['repo']['has_discussions']", "root['pull_request']['base']['repo']['forks_count']", "root['pull_request']['base']['repo']['mirror_url']", "root['pull_request']['base']['repo']['archived']", "root['pull_request']['base']['repo']['disabled']", "root['pull_request']['base']['repo']['open_issues_count']", "root['pull_request']['base']['repo']['license']", "root['pull_request']['base']['repo']['allow_forking']", "root['pull_request']['base']['repo']['is_template']", "root['pull_request']['base']['repo']['web_commit_signoff_required']", "root['pull_request']['base']['repo']['topics']", "root['pull_request']['base']['repo']['visibility']", "root['pull_request']['base']['repo']['forks']", "root['pull_request']['base']['repo']['open_issues']", "root['pull_request']['base']['repo']['watchers']", "root['pull_request']['base']['repo']['default_branch']"],
 'type_changes': {"root['review']['body']": {'new_type': <class 'str'>,
                                             'old_type': <class 'NoneType'>}}}

=== PullRequestReviewCommentEvent ===
{'dictionary_item_added': ["root['comment']['in_reply_to_id']"],
 'dictionary_item_removed': ["root['comment']['author_association']", "root['comment']['start_line']", "root['comment']['original_start_line']", "root['comment']['start_side']", "root['comment']['line']", "root['comment']['original_line']", "root['comment']['side']", "root['pull_request']['node_id']", "root['pull_request']['html_url']", "root['pull_request']['diff_url']", "root['pull_request']['patch_url']", "root['pull_request']['issue_url']", "root['pull_request']['state']", "root['pull_request']['locked']", "root['pull_request']['title']", "root['pull_request']['user']", "root['pull_request']['body']", "root['pull_request']['created_at']", "root['pull_request']['updated_at']", "root['pull_request']['closed_at']", "root['pull_request']['merged_at']", "root['pull_request']['merge_commit_sha']", "root['pull_request']['assignee']", "root['pull_request']['assignees']", "root['pull_request']['requested_reviewers']", "root['pull_request']['requested_teams']", "root['pull_request']['labels']", "root['pull_request']['milestone']", "root['pull_request']['draft']", "root['pull_request']['commits_url']", "root['pull_request']['review_comments_url']", "root['pull_request']['review_comment_url']", "root['pull_request']['comments_url']", "root['pull_request']['statuses_url']", "root['pull_request']['_links']", "root['pull_request']['author_association']", "root['pull_request']['auto_merge']", "root['pull_request']['active_lock_reason']", "root['pull_request']['head']['label']", "root['pull_request']['head']['user']", "root['pull_request']['head']['repo']['node_id']", "root['pull_request']['head']['repo']['full_name']", "root['pull_request']['head']['repo']['private']", "root['pull_request']['head']['repo']['owner']", "root['pull_request']['head']['repo']['html_url']", "root['pull_request']['head']['repo']['description']", "root['pull_request']['head']['repo']['fork']", "root['pull_request']['head']['repo']['forks_url']", "root['pull_request']['head']['repo']['keys_url']", "root['pull_request']['head']['repo']['collaborators_url']", "root['pull_request']['head']['repo']['teams_url']", "root['pull_request']['head']['repo']['hooks_url']", "root['pull_request']['head']['repo']['issue_events_url']", "root['pull_request']['head']['repo']['events_url']", "root['pull_request']['head']['repo']['assignees_url']", "root['pull_request']['head']['repo']['branches_url']", "root['pull_request']['head']['repo']['tags_url']", "root['pull_request']['head']['repo']['blobs_url']", "root['pull_request']['head']['repo']['git_tags_url']", "root['pull_request']['head']['repo']['git_refs_url']", "root['pull_request']['head']['repo']['trees_url']", "root['pull_request']['head']['repo']['statuses_url']", "root['pull_request']['head']['repo']['languages_url']", "root['pull_request']['head']['repo']['stargazers_url']", "root['pull_request']['head']['repo']['contributors_url']", "root['pull_request']['head']['repo']['subscribers_url']", "root['pull_request']['head']['repo']['subscription_url']", "root['pull_request']['head']['repo']['commits_url']", "root['pull_request']['head']['repo']['git_commits_url']", "root['pull_request']['head']['repo']['comments_url']", "root['pull_request']['head']['repo']['issue_comment_url']", "root['pull_request']['head']['repo']['contents_url']", "root['pull_request']['head']['repo']['compare_url']", "root['pull_request']['head']['repo']['merges_url']", "root['pull_request']['head']['repo']['archive_url']", "root['pull_request']['head']['repo']['downloads_url']", "root['pull_request']['head']['repo']['issues_url']", "root['pull_request']['head']['repo']['pulls_url']", "root['pull_request']['head']['repo']['milestones_url']", "root['pull_request']['head']['repo']['notifications_url']", "root['pull_request']['head']['repo']['labels_url']", "root['pull_request']['head']['repo']['releases_url']", "root['pull_request']['head']['repo']['deployments_url']", "root['pull_request']['head']['repo']['created_at']", "root['pull_request']['head']['repo']['updated_at']", "root['pull_request']['head']['repo']['pushed_at']", "root['pull_request']['head']['repo']['git_url']", "root['pull_request']['head']['repo']['ssh_url']", "root['pull_request']['head']['repo']['clone_url']", "root['pull_request']['head']['repo']['svn_url']", "root['pull_request']['head']['repo']['homepage']", "root['pull_request']['head']['repo']['size']", "root['pull_request']['head']['repo']['stargazers_count']", "root['pull_request']['head']['repo']['watchers_count']", "root['pull_request']['head']['repo']['language']", "root['pull_request']['head']['repo']['has_issues']", "root['pull_request']['head']['repo']['has_projects']", "root['pull_request']['head']['repo']['has_downloads']", "root['pull_request']['head']['repo']['has_wiki']", "root['pull_request']['head']['repo']['has_pages']", "root['pull_request']['head']['repo']['has_discussions']", "root['pull_request']['head']['repo']['forks_count']", "root['pull_request']['head']['repo']['mirror_url']", "root['pull_request']['head']['repo']['archived']", "root['pull_request']['head']['repo']['disabled']", "root['pull_request']['head']['repo']['open_issues_count']", "root['pull_request']['head']['repo']['license']", "root['pull_request']['head']['repo']['allow_forking']", "root['pull_request']['head']['repo']['is_template']", "root['pull_request']['head']['repo']['web_commit_signoff_required']", "root['pull_request']['head']['repo']['topics']", "root['pull_request']['head']['repo']['visibility']", "root['pull_request']['head']['repo']['forks']", "root['pull_request']['head']['repo']['open_issues']", "root['pull_request']['head']['repo']['watchers']", "root['pull_request']['head']['repo']['default_branch']", "root['pull_request']['base']['label']", "root['pull_request']['base']['user']", "root['pull_request']['base']['repo']['node_id']", "root['pull_request']['base']['repo']['full_name']", "root['pull_request']['base']['repo']['private']", "root['pull_request']['base']['repo']['owner']", "root['pull_request']['base']['repo']['html_url']", "root['pull_request']['base']['repo']['description']", "root['pull_request']['base']['repo']['fork']", "root['pull_request']['base']['repo']['forks_url']", "root['pull_request']['base']['repo']['keys_url']", "root['pull_request']['base']['repo']['collaborators_url']", "root['pull_request']['base']['repo']['teams_url']", "root['pull_request']['base']['repo']['hooks_url']", "root['pull_request']['base']['repo']['issue_events_url']", "root['pull_request']['base']['repo']['events_url']", "root['pull_request']['base']['repo']['assignees_url']", "root['pull_request']['base']['repo']['branches_url']", "root['pull_request']['base']['repo']['tags_url']", "root['pull_request']['base']['repo']['blobs_url']", "root['pull_request']['base']['repo']['git_tags_url']", "root['pull_request']['base']['repo']['git_refs_url']", "root['pull_request']['base']['repo']['trees_url']", "root['pull_request']['base']['repo']['statuses_url']", "root['pull_request']['base']['repo']['languages_url']", "root['pull_request']['base']['repo']['stargazers_url']", "root['pull_request']['base']['repo']['contributors_url']", "root['pull_request']['base']['repo']['subscribers_url']", "root['pull_request']['base']['repo']['subscription_url']", "root['pull_request']['base']['repo']['commits_url']", "root['pull_request']['base']['repo']['git_commits_url']", "root['pull_request']['base']['repo']['comments_url']", "root['pull_request']['base']['repo']['issue_comment_url']", "root['pull_request']['base']['repo']['contents_url']", "root['pull_request']['base']['repo']['compare_url']", "root['pull_request']['base']['repo']['merges_url']", "root['pull_request']['base']['repo']['archive_url']", "root['pull_request']['base']['repo']['downloads_url']", "root['pull_request']['base']['repo']['issues_url']", "root['pull_request']['base']['repo']['pulls_url']", "root['pull_request']['base']['repo']['milestones_url']", "root['pull_request']['base']['repo']['notifications_url']", "root['pull_request']['base']['repo']['labels_url']", "root['pull_request']['base']['repo']['releases_url']", "root['pull_request']['base']['repo']['deployments_url']", "root['pull_request']['base']['repo']['created_at']", "root['pull_request']['base']['repo']['updated_at']", "root['pull_request']['base']['repo']['pushed_at']", "root['pull_request']['base']['repo']['git_url']", "root['pull_request']['base']['repo']['ssh_url']", "root['pull_request']['base']['repo']['clone_url']", "root['pull_request']['base']['repo']['svn_url']", "root['pull_request']['base']['repo']['homepage']", "root['pull_request']['base']['repo']['size']", "root['pull_request']['base']['repo']['stargazers_count']", "root['pull_request']['base']['repo']['watchers_count']", "root['pull_request']['base']['repo']['language']", "root['pull_request']['base']['repo']['has_issues']", "root['pull_request']['base']['repo']['has_projects']", "root['pull_request']['base']['repo']['has_downloads']", "root['pull_request']['base']['repo']['has_wiki']", "root['pull_request']['base']['repo']['has_pages']", "root['pull_request']['base']['repo']['has_discussions']", "root['pull_request']['base']['repo']['forks_count']", "root['pull_request']['base']['repo']['mirror_url']", "root['pull_request']['base']['repo']['archived']", "root['pull_request']['base']['repo']['disabled']", "root['pull_request']['base']['repo']['open_issues_count']", "root['pull_request']['base']['repo']['license']", "root['pull_request']['base']['repo']['allow_forking']", "root['pull_request']['base']['repo']['is_template']", "root['pull_request']['base']['repo']['web_commit_signoff_required']", "root['pull_request']['base']['repo']['topics']", "root['pull_request']['base']['repo']['visibility']", "root['pull_request']['base']['repo']['forks']", "root['pull_request']['base']['repo']['open_issues']", "root['pull_request']['base']['repo']['watchers']", "root['pull_request']['base']['repo']['default_branch']"]}

=== PushEvent ===
{'dictionary_item_removed': ["root['size']", "root['distinct_size']", "root['commits']"]}

Notebook Ref: https://www.oso.xyz/oso/gharchive-monitoring-v2

kenyiu avatar Dec 05 '25 15:12 kenyiu

The number of rows of the models that are the root cause for the zero-ed events.

model referenced in # of relevant null from 2025-10-6 to 2025-12-6 # of null for nstg_github__eventsov daily avg # of null for nov
stg_github__push_events stg_github__commits 126731001 75069424 2502314
stg_github__events stg_github__pull_request 12014787 6653352 221778
stg_github__events stg_github__comments 2597544 1479303 49310

kenyiu avatar Dec 08 '25 15:12 kenyiu