test-infra icon indicating copy to clipboard operation
test-infra copied to clipboard

Update disabled issues from other repos

Open huydhn opened this issue 1 year ago • 5 comments

This is to capture disabled and unstable issues from other repos, i.e. https://github.com/pytorch/executorch/issues/3264. The easiest way is to gather all these issues in the same file instead of having one per repo. There is a Rockset query https://github.com/pytorch/test-infra/blob/main/torchci/rockset/commons/__sql/issue_query.sql that we use to query the issue by their labels, but it already covers all repos in the org.

huydhn avatar Apr 24 '24 22:04 huydhn

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated (UTC)
torchci ⬜️ Ignored (Inspect) Apr 24, 2024 10:36pm

vercel[bot] avatar Apr 24 '24 22:04 vercel[bot]

does the handling of this exist inside filter test configs?

clee2000 avatar Apr 24 '24 23:04 clee2000

does the handling of this exist inside filter test configs?

The JSON file itself is in the correct format. I run the script locally and get:

{
  "trunk / test-coreml-delegate / macos-job": [
    "huydhn",
    "3264",
    "https://github.com/pytorch/executorch/issues/3264",
    "trunk",
    "test-coreml-delegate",
    "macos-job"
  ]
}

We don't have a repo check on filter test configs script atm, but I think that could be patched up properly later if we need to migrate filter config to other repos because there is already logic to check against the workflow and job name in https://github.com/pytorch/pytorch/blob/main/.github/scripts/filter_test_configs.py#L361-L366. The script is also available in PyTorch and nowhere else. So as long as there is no job called test-coreml-delegate in any other repos, it should be ok I think. The job name is pretty unique, hopefully this wouldn't come back and bite me later lol.

huydhn avatar Apr 24 '24 23:04 huydhn

I guess adding on to that, does this actually need to be changed given that no other repos have filter test configs? I can't figure out what else consumes this list. Is this in preparation for a future change?

clee2000 avatar Apr 25 '24 21:04 clee2000

I guess adding on to that, does this actually need to be changed given that no other repos have filter test configs? I can't figure out what else consumes this list. Is this in preparation for a future change?

My initial thought was to consume this file in Dr.CI https://github.com/pytorch/test-infra/pull/5131 and mergebot https://github.com/pytorch/pytorch/pull/124965. But then, I go with the route to call await fetchIssuesByLabel("unstable") to get the list of unstable issue instead because HUD is using that function. That approach looks better because all components are using the same way. That function queries Rockset instead, so the JSON file becomes redundant. Maybe we will find an use case for this later, but nothing is depending on this atm. I'll keep this open for a bit longer in case I miss anything. After that, we can close this if this is not needed.

huydhn avatar Apr 26 '24 01:04 huydhn