authd icon indicating copy to clipboard operation
authd copied to clipboard

Make SSH integration tests easier to debug

Open adombeck opened this issue 3 months ago • 3 comments

Changes for improved debuggability, test execution time, and maintainability, including:

  • Lots of improvements to the logs printed by the tests.
  • Avoid unnecessary rebuilds
  • Refactorings

UDENG-8137

adombeck avatar Aug 28 '25 12:08 adombeck

Codecov Report

:x: Patch coverage is 80.26316% with 15 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 86.77%. Comparing base (42c81e0) to head (44d38b9). :warning: Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/testlog/testlog.go 78.57% 15 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1046      +/-   ##
==========================================
- Coverage   87.78%   86.77%   -1.02%     
==========================================
  Files          89       90       +1     
  Lines        6150     6222      +72     
  Branches      111      111              
==========================================
  Hits         5399     5399              
- Misses        695      750      +55     
- Partials       56       73      +17     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Sep 01 '25 14:09 codecov-commenter

@3v1n0 there is a bit of back and forth with how we launch sshd and how we print and store its logs - I'm sorry about that, but there are too many merge conflicts when I try to squash those commit.

This PR contains a lot of orthogonal changes, which don't necessarily have to be merged all at once. So if there's something that needs more discussion, we can try to move it to another branch (if it doesn't cause too many merge conflicts).

adombeck avatar Sep 03 '25 08:09 adombeck

Rebased on main and resolved conflicts.

@3v1n0, I just spent over an hour resolving conflicts, it would be great if you could review this soon to avoid further merge conflicts :)

adombeck avatar Nov 17 '25 18:11 adombeck

Since Marco doesn't seem to find the time to review this, do you think you could take over the review @denisonbarbosa? There are some nice improvements here which I would also like to use in other tests.

adombeck avatar Dec 02 '25 19:12 adombeck

Thanks a lot for the review!

adombeck avatar Dec 03 '25 12:12 adombeck

This error was weird, we miss a golden?

      ===== VHS tape "simple_auth_locks_unlocks" finished in 21.674s =====
      
      
      ssh_test.go:545: 
          	Error Trace:	/home/runner/work/authd/authd/internal/testutils/golden/golden.go:248
          	            				/home/runner/work/authd/authd/internal/testutils/golden/golden.go:99
          	            				/home/runner/work/authd/authd/pam/integration-tests/ssh_test.go:545
          	Error:      	Received unexpected error:
          	            	open /home/runner/work/authd/authd/pam/integration-tests/testdata/golden/TestSSHAuthenticate/Authenticate_user_locks_and_unlocks_it_with_shared_sshd: no such file or directory
          	Test:       	TestSSHAuthenticate/Authenticate_user_locks_and_unlocks_it_with_shared_sshd
          	Messages:   	Cannot read golden file /home/runner/work/authd/authd/pam/integration-tests/testdata/golden/TestSSHAuthenticate/Authenticate_user_locks_and_unlocks_it_with_shared_sshd

3v1n0 avatar Dec 04 '25 20:12 3v1n0

This error was weird, we miss a golden?

fixed

adombeck avatar Dec 04 '25 21:12 adombeck