magma icon indicating copy to clipboard operation
magma copied to clipboard

chore(ci): Run the LTE integ_tests with Bazel

Open vktng opened this issue 2 years ago • 8 comments

Signed-off-by: Krisztián Varga [email protected] Co-authored-by: Lars Kreutzer [email protected] Co-authored-by: Krisztián Varga [email protected]

Summary

In the LTE integ test Bazel workflow the integration tests are now run with Bazel. This closes #13381

Test Plan

For testing see: https://github.com/magma/magma/issues/13381

Final config run (With some unrelated test failures, which happen at master too.):

  • https://github.com/vktng/magma/runs/7997130758?check_suite_focus=true
  • https://github.com/LKreutzer/magma/runs/8010104491?check_suite_focus=true
  • https://github.com/vktng/magma/runs/8067865104?check_suite_focus=true

Run without sudo tests:

  • https://github.com/vktng/magma/runs/8067945830?check_suite_focus=true

Runs with good list: with sudo tests:

  • https://github.com/vktng/magma/runs/8074534983?check_suite_focus=true
    • 2h 3m with 1 failing test
    • Bazel
  • https://github.com/vktng/magma/runs/8089413333?check_suite_focus=true
    • Bazel
    • flaky test removed
    • publishing results
  • https://github.com/vktng/magma/runs/8074530090?check_suite_focus=true
    • Make

Without sudo tests:

  • https://github.com/vktng/magma/runs/8074913942?check_suite_focus=true

New runs after rebase:

  • https://github.com/vktng/magma/runs/8110989651?check_suite_focus=true
  • https://github.com/vktng/magma/runs/8130071536?check_suite_focus=true
    • with disable checksumming

Run with fix from https://github.com/magma/magma/pull/13803: https://github.com/LKreutzer/magma/runs/8152381609?check_suite_focus=true

Additional Information

  • [ ] This change is backwards-breaking

vktng avatar Aug 17 '22 14:08 vktng

Thanks for opening a PR! :100:

A couple initial guidelines

Howto

  • Reviews. The "Reviewers" listed for this PR are the Magma maintainers who will shepherd it.
  • Checks. All required CI checks must pass before merge.
  • Merge. Once approved and passing CI checks, use the ready2merge label to indicate the maintainers can merge your PR.

More info

Please take a moment to read through the Magma project's

If this is your first Magma PR, also consider reading

github-actions[bot] avatar Aug 17 '22 14:08 github-actions[bot]

feg-workflow

    2 files  203 suites   39s :stopwatch: 374 tests 374 :heavy_check_mark: 0 :zzz: 0 :x: 388 runs  388 :heavy_check_mark: 0 :zzz: 0 :x:

Results for commit 45ffc1da.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Aug 17 '22 14:08 github-actions[bot]

dp-workflow

14 tests   14 :heavy_check_mark:  2m 10s :stopwatch:   1 suites    0 :zzz:   1 files      0 :x:

Results for commit 45ffc1da.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Aug 17 '22 15:08 github-actions[bot]

agw-workflow

615 tests   611 :heavy_check_mark:  3m 47s :stopwatch:     2 suites      4 :zzz:     2 files        0 :x:

Results for commit 45ffc1da.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Aug 17 '22 15:08 github-actions[bot]

Oops! Looks like you failed the Python Format Check.

Howto

:recycle: Updated: :white_check_mark: The check is passing the Python Format Check after the last commit.

github-actions[bot] avatar Aug 18 '22 15:08 github-actions[bot]

Oops! Looks like you failed the DCO check. Be sure to sign all your commits.

Howto

:recycle: Updated: :white_check_mark: The check is passing the DCO check after the last commit.

github-actions[bot] avatar Aug 23 '22 09:08 github-actions[bot]

from offline discussion: runtime on forks seems to be long - is the reason that tests are failing (i.e., timeouts etc)? This should be analyzed.

nstng avatar Aug 29 '22 10:08 nstng

Rebased on fabfile and workflow changes

LKreutzer avatar Sep 14 '22 11:09 LKreutzer

Tests will not be executed with bazel. This has performance issues:

  • hypothesis is that the large pythonpath generated by bazel is causing this.
  • integ tests do not benefit from bazel caching as they need to be executed every time - and a relatively small artifacts with a long run-time
  • see also https://github.com/bazelbuild/rules_python/issues 311

Bute the environment for test execution will be build with bazel - see #14286.

nstng avatar Nov 02 '22 06:11 nstng