aardvark-dns icon indicating copy to clipboard operation
aardvark-dns copied to clipboard

[skip-ci] Packit/TMT: simplify tests

Open lsm5 opened this issue 1 year ago • 8 comments

Use idiomatic packit to configure podman-next copr repos for upstream tests and simplify test script.

lsm5 avatar Jul 31 '24 18:07 lsm5

Tests failed. @containers/packit-build please check.

test 5 is run on rhel-9 and c9s in this run, but being skipped on c10s.

lsm5 avatar Jul 31 '24 18:07 lsm5

@Luap99 @edsantiago any idea about the rhel9 / c9s failures?

lsm5 avatar Aug 02 '24 17:08 lsm5

@Luap99 @edsantiago any idea about the rhel9 / c9s failures?

# #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
# #|     FAIL: server used TCP
# #| expected: =~ '\(TCP\)'
# #|   actual:    '; <<>> DiG 9.16.23-RH <<>> +tcp google.com @10.45.253.1'
# #|         >    ';; global options: +cmd'
# #|         >    ';; Got answer:'
# #|         >    ';; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56432'
# #|         >    ';; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1'
# #|         >    ';; OPT PSEUDOSECTION:'
# #|         >    '; EDNS: version: 0, flags:; udp: 4096'
# #|         >    ';; QUESTION SECTION:'
# #|         >    ';google.com.			IN	A'
# #|         >    ';; ANSWER SECTION:'
# #|         >    'google.com.		179	IN	A	172.217.2.46'
# #|         >    ';; Query time: 9 msec'
# #|         >    ';; SERVER: 10.45.253.1#53(10.45.253.1)'
# #|         >    ';; WHEN: Fri Aug 02 16:18:11 UTC 2024'
# #|         >    ';; MSG SIZE  rcvd: 55'
# #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

the dig version doesn't output TCP in the output so the test has no way to guarantee that we actually used tcp other than the +tcp arg (which well may be good enough), on newer versions the SERVER lines contains either (TCP) or (UDP)

I like to keep the TCP output check if possible because it is easy remove the +tcp argument by accident but I don't see a good way to work around that either other than removing this check

Luap99 avatar Aug 02 '24 17:08 Luap99

The (TCP|UDP) output seems to be very new, in bind 9.18 or so (it is not in bind 9.16). I'm guessing that RHEL9 has too old a version of bind-utils. @Luap99 perhaps a better safeguard might be

    assert "$output" !~ "SERVER.*UDP"

edsantiago avatar Aug 02 '24 17:08 edsantiago

@Luap99 RE: https://github.com/containers/aardvark-dns/pull/519#issuecomment-2379676552 did you mean it worked on cs10 for you? Going by the current test run, looks like installing bats from source behaves just the same on cs10.

lsm5 avatar Sep 30 '24 11:09 lsm5

@Luap99 RE: #519 (comment) did you mean it worked on cs10 for you? Going by the current test run, looks like installing bats from source behaves just the same on cs10.

yes, I had working tests on a centos stream 10 VM with the bats 1.11.

However even with running tmt locally the setup was somewhat different as there was no bats or slirp4netns to be found so I am not sure how we get it in the CI but not locally I used tmt -c initiator=packit try centos-stream-10@virtual and there used the interactive terminal and to install the missing deps and then I got it working.

But as said it also failed to run any test there not just the one so I really do not know what is different inside testing farm. I guess I have to try with the exact steps from the tmt-reproducer output.

Luap99 avatar Sep 30 '24 11:09 Luap99

CentOS 9 and 10 work now. PTAL.

I did see the three subnets test fail once on centos 9 stream aarch64. Flake probably?

lsm5 avatar Oct 31 '24 14:10 lsm5

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lsm5, Luap99

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci[bot] avatar Nov 01 '24 10:11 openshift-ci[bot]