aqa-tests icon indicating copy to clipboard operation
aqa-tests copied to clipboard

CryptoTests are reporting GssApiMechanismTests and SaslServerFactoryTests as passed, where it should be skipped

Open judovana opened this issue 1 year ago • 3 comments

Those two tests - SaslServerFactoryTests and GssApiMechanismTests are relying on remote kerberos server, which is disabled in Aqa-tests by SKIP_AGENT_TESTS/cryptotests.skipAgentTests

However setting up this property/variable is making the tests silently pass, instead of ignoring them. I will be fixing this in upstream today. It thsould not affect the aqavit runs, onluy those two tests will move from passed to ignored.

However I would like to enable those two tests. The kerberos server it needs, have accidentaly same setup as kerberos server for tck. Is there any available ?

In both cases, I would like to improve a bit the aqavit crypto test targets, so they allow the passing of both AGENT_HOSTNAME and SKIP_AGENT_TESTS inside, so anybody who have kerbeors server can point Cryptotest targets to it.

In addition, this server can be setup locally by morevoer simple script - if all depndences are satisfied. Thus I was thinking, about creating an external aqa-tests test, which would run only those two tests, and will prepare the local kerberos server inside the container. WDYT @smlambert @llxia @zzambers @annaibm @pshipton?

judovana avatar Apr 19 '24 13:04 judovana

The https://github.com/rh-openjdk/CryptoTest/pull/68 is merged, so the two tests should now disapear fro all runs which do have agent enabled.

judovana avatar Apr 19 '24 13:04 judovana

Due to legal requirements, our TCK tests run in private and the kerberos server is also not publicly available.

We had previously talked about adding a feature to run a server in a container and made available for the duration of tests that require it.

Let's design an approach that can be used for many scenarios where any test target that needs a containerized server can use the same method.

smlambert avatar Apr 19 '24 13:04 smlambert

The five tests now shoudl be properly ignored. Will think about the kerberos localhost a bit

judovana avatar Apr 24 '24 16:04 judovana