test-infra icon indicating copy to clipboard operation
test-infra copied to clipboard

DO NOT CLOSE IT / GitHub Workflow Runner Determinator

Open DanilBaibak opened this issue 1 year ago • 15 comments

In the FIRST comment you can specify the configuration for each experiment.

The format is:

  • Above the line break, you have yaml formatted text listing all experiments and their configurations
  • Below the line break, you list all the people who should be opted into any experiment, followed by a comma separated list of all experiments they should be opted into.
  • If you comment out a person (by prefixing their line with #) then that line will have no effect

Anyone opted in will always have the given experiment enabled. For everyone else, they will have a rollout_perc chance of having the experiment enabled on their workflow. Note that the experiment is always enabled/disabled at the workflow level, not the job level.

This configuration is parsed by _runner-determinator.yml workflow on pytorch/pytorch.

DanilBaibak avatar Apr 25 '24 09:04 DanilBaibak

experiments:
    lf:
       rollout_perc: 95
       all_branches: true

Opted-in:

@ZainRizvi,lf @wdvr,lf @atalman,lf @seemethere,lf @botmethere,lf @clee2000,lf @huydhn,lf @izaitsevfb,lf @jeanschmidt,lf @PaliC,lf @kit1980,lf @malfet,lf @zxiiro,lf

DanilBaibak avatar Apr 25 '24 09:04 DanilBaibak

both

DanilBaibak avatar May 03 '24 12:05 DanilBaibak

Should we close this?

kit1980 avatar May 31 '24 18:05 kit1980

No, pls don't close the issue. It will be used for dynamic rollout while we are migrating from our runners to the linux foundation runners.

DanilBaibak avatar Jun 03 '24 12:06 DanilBaibak

Removed myself from the list, as LF runners queue is huge.. Also, why everyone need to be mentioned twice?

malfet avatar Jun 26 '24 13:06 malfet

Removed myself from the list, as LF runners queue is huge.. Also, why everyone need to be mentioned twice?

PR pytorch/pytorch#129246 changed it so that we are using @ prefixed usernames. So I think the non-@ prefix ones no longer need to be there.

zxiiro avatar Jun 26 '24 13:06 zxiiro

Looks like pytorch/pytorch#129462 introduced a regression where it no longer properly checked the @ prefixed usernames. I have a fix on that but can't open a PR as it depends on pytorch/pytorch#129500 getting merged first. Just waiting for PR checks to complete and I'll try to get it in and open a PR for the @ fix.

zxiiro avatar Jun 26 '24 18:06 zxiiro