mina icon indicating copy to clipboard operation
mina copied to clipboard

Test reporter in Lucy (berkeley)

Open dkijania opened this issue 1 year ago • 14 comments

Explain your changes:

Introduced test reported module, which is responsible for converting test executive log into junit.xml format which then can be consumed by buildkite test analytics module: https://buildkite.com/docs/test-analytics#main. I had to make some changes in Lucy code as well to correctly determined what is test result. For this purpose I've added new log level (Span), which is a common way to inform reader that about inner scope of logs.

Currently only junit.xml is supported (preferred was buildkite json file, but buildkite api has issues on uploading.. resolving it with buildkite support team).

Link to rfc: https://github.com/o1-labs/rfcs/pull/15

Checklist:

  • [ ] Modified the current draft of release notes with details on what is completed or incomplete within this project
  • [ ] Document code purpose, how to use it
    • Mention expected invariants, implicit constraints
  • [ ] Tests were added for the new behavior
    • Document test purpose, significance of failures
    • Test names should reflect their purpose
  • [ ] All tests pass (CI will check this if you didn't)
  • [ ] Serialized types are in stable-versioned modules
  • [ ] Does this close issues? List them

dkijania avatar Apr 11 '23 11:04 dkijania

!ci-build-me

dkijania avatar Jun 01 '23 08:06 dkijania

!ci-build-me

dkijania avatar Jun 15 '23 16:06 dkijania

!ci-build-me

dkijania avatar Jun 16 '23 13:06 dkijania

!ci-build-me

dkijania avatar Jun 26 '23 09:06 dkijania

!ci-build-me

dkijania avatar Jun 26 '23 17:06 dkijania

!ci-build-me

dkijania avatar Jun 27 '23 07:06 dkijania

!ci-build-me

dkijania avatar Jul 04 '23 06:07 dkijania

!ci-build-me

dkijania avatar Jul 06 '23 20:07 dkijania

!ci-build-me

dkijania avatar Jul 06 '23 21:07 dkijania

!ci-build-me

dkijania avatar Jul 10 '23 11:07 dkijania

!ci-build-me

dkijania avatar Oct 23 '23 11:10 dkijania

!ci-build-me

dkijania avatar Oct 24 '23 21:10 dkijania

!ci-build-me

dkijania avatar Oct 25 '23 19:10 dkijania

This PR is required to satisfy test executive reporting to buildkite. Thanks to test reported we can upload test results to bk and it will contribute to overall test execution graph in test module in bk

dkijania avatar Feb 02 '24 14:02 dkijania