chromium-bidi icon indicating copy to clipboard operation
chromium-bidi copied to clipboard

`test_input_performActions* ` e2e test is flaky

Open sadym-chromium opened this issue 2 years ago • 4 comments

e2e tests of input are flaky. Example:

test_input_performActionsEmitsDragging: https://github.com/GoogleChromeLabs/chromium-bidi/actions/runs/6448243919/job/17509797173?pr=1403

Although the output is really hard to read, the difference is expected and missing the following part at the beginning:

      {
        type: 'object',
        value: [
          [
            'event',
            {
              type: 'string',
              value: 'mousemove',
            },
          ],
          [
            'button',
            {
              type: 'number',
              value: 0,
            },
          ],
          [
            'buttons',
            {
              type: 'number',
              value: 0,
            },
          ],
          [
            'clientX',
            {
              type: 'number',
              value: 630,
            },
          ],
          [
            'clientY',
            {
              type: 'number',
              value: 336,
            },
          ],
        ],
      },

Another example:

test_input_performActionsEmitsPointerEvents: Example: https://github.com/GoogleChromeLabs/chromium-bidi/actions/runs/6548857552/job/17784481994?pr=1443 Related: https://github.com/GoogleChromeLabs/chromium-bidi/issues/1407

sadym-chromium avatar Oct 09 '23 09:10 sadym-chromium

This is a result of headful mode. The solution is to disable the test in headful.

jrandolf-google avatar Oct 09 '23 11:10 jrandolf-google

This is a result of headful mode. The solution is to disable the test in headful.

Cannot we make the test more flexible, so that it tolerates the fluctuations?

sadym-chromium avatar Oct 09 '23 11:10 sadym-chromium

friendly ping. This causes false-negative CI runs almost daily

sadym-chromium avatar Oct 13 '23 08:10 sadym-chromium

another example: https://github.com/GoogleChromeLabs/chromium-bidi/actions/runs/6509030633/job/17679618738

sadym-chromium avatar Oct 13 '23 14:10 sadym-chromium

The issue addressed, expectations do not tolerate flakiness anymore

sadym-chromium avatar May 27 '24 09:05 sadym-chromium