Confusing response to test session and its result state

[AlexThurston]

environment DEMO

testSessionId 566292

vsId 2720997 and 2720998

Algorithm registration N/A

Endpoint in which the error is experienced https://demo.acvts.nist.gov/acvp/v1/testSessions/566292 GET https://demo.acvts.nist.gov/acvp/v1/testSessions/566292/results GET

Expected behaviour I'll start off by saying that the test session in question is expired. This might be the cause of the root issue but I believe it is still an issue none the less.

Reading the test session https://demo.acvts.nist.gov/acvp/v1/testSessions/566292 GET responds with:

{
  "createdOn": "2024-12-05T10:01:45Z",
  "expiresOn": "2025-01-05T07:10:28Z",
  "isSample": true,
  "passed": false,
  "publishable": false,
  "url": "/acvp/v1/testSessions/566292",
  "vectorSetUrls": [
    "/acvp/v1/testSessions/566292/vectorSets/2720997",
    "/acvp/v1/testSessions/566292/vectorSets/2720998"
  ]
}

Notably, ACVTS is reporting that the test session is not passed.

If I read the results of the test session https://demo.acvts.nist.gov/acvp/v1/testSessions/566292/results GET, the response is:

{
  "passed": false,
  "results": [
    {
      "status": "passed",
      "vectorSetUrl": "/acvp/v1/testSessions/566292/vectorSets/2720997"
    },
    {
      "status": "passed",
      "vectorSetUrl": "/acvp/v1/testSessions/566292/vectorSets/2720998"
    }
  ]
}

As you can see both vector sets are being registered as passed but similar to the GET on the test session the overall passed is still false which doesn't seem correct.

As I mentioned above, I'm going to guess that this is because the test session is expired, but even still, I'd expect that passed state to be independent from the expiry.

[cpu]

FWIW I've also run into this with unexpired sessions, and found this previous issue https://github.com/usnistgov/ACVP-Server/issues/183 that seemed to indicate it was a server-side data race 🤷

[jbrock24]

I'll start off by saying that the test session in question is expired. This might be the cause of the root issue but I believe it is still an issue none the less.

In this scenario, this is the condition that caused the TS to have passed = false. We are unable to say that a TS expired based on the spec limitations, but when it is expired, it will display the TS as passed=false. Thank you for the question @AlexThurston , it led to some great info exchange and discussion. Please let me know if you have any other questions.

[jbrock24]

Related to Github/ACVP issue.

[jbrock24]

Hi @cpu - This has been added to our future plans, but will not be resolved here. I am going to close this out, but feel free to reopen with any follow-up questions. Thanks!