oras icon indicating copy to clipboard operation
oras copied to clipboard

test: setup E2E tests for ORAS

Open qweeah opened this issue 3 years ago • 1 comments

This PR introduces E2E testing based on ginkgo and gomega. E2E tests will be run against ORAS binary built during Build CLI step and latest oras-distribution container with below changes:

  • Build CI action: Added start & stop steps for backend distribution
  • Test: added E2E tests running together with unit tests during make test
    • Two test suites: TestORASCommands for single command tests and TestORASScenarios for scenario-based tests
    • Testing utilities: added command execution helper types and funcs, as well as a output matcher package.

Now the PR also contains sample e2e tests for:

  1. login & logout commands, as well as failure checks for all commands after logging out .
  2. helper doc check for manifest fetch command

To run the tests, a guide can be found in here. related to #523

qweeah avatar Aug 29 '22 01:08 qweeah

Codecov Report

Merging #528 (c25eb1e) into main (9a014d6) will increase coverage by 5.92%. The diff coverage is 98.29%.

@@            Coverage Diff             @@
##             main     #528      +/-   ##
==========================================
+ Coverage   70.72%   76.65%   +5.92%     
==========================================
  Files          11       14       +3     
  Lines         427      544     +117     
==========================================
+ Hits          302      417     +115     
- Misses        101      102       +1     
- Partials       24       25       +1     
Impacted Files Coverage Δ
test/e2e/scenario/oras_artifact.go 96.87% <96.87%> (ø)
test/e2e/command/manifest.go 100.00% <100.00%> (ø)
test/e2e/scenario/auth.go 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov-commenter avatar Aug 29 '22 01:08 codecov-commenter

Closing this, will add another PR for testing utilities

qweeah avatar Oct 10 '22 00:10 qweeah