ACVP-Server icon indicating copy to clipboard operation
ACVP-Server copied to clipboard

Server health endpoint results missing status value

Open mikeboulet opened this issue 9 months ago • 3 comments

The /health endpoint is not returning a "status" key. This seems to happen every so often with this endpoint.

Here are the results from the endpoint from 6:05 EST this morning.

{
  "serverVersion": "v1.1.0.38",
  "details": [
    {
      "key": "testSessionProcessing",
      "description": "The TestSession internal processing load status.",
      "data": {
        "healthStatusDefinitions": {
          "Healthy": "Oldest pending TestSession is \u003c 1 hours old.",
          "Degraded": "Oldest pending TestSession is \u003e 1 hours old.",
          "Unhealthy": "Oldest pending TestSession is \u003e 4 hours old."
        }
      }
    }
  ]
}

mikeboulet avatar Apr 01 '25 13:04 mikeboulet

Thanks for mentioning this @mikeboulet. From a skim of related GitHub issues, there could be a correlation between high server load and status missing. Demo was working through a backlog of test vectors starting around midnight. There were 3,517 tasks queued at 12:30am and by 5am there were still 1003 tasks queued. When we had a chance to check the health endpoint around 10am, the endpoint was again returning status with a value of "Healthy". Something is obviously not working as it should. We'll look into what may be causing this.

livebe01 avatar Apr 02 '25 14:04 livebe01

Hi @mikeboulet, I have fixed this issue and the change will go out with a future release. We'll post here once that's been completed. Thanks for the Issue creation!

jbrock24 avatar Aug 20 '25 20:08 jbrock24

The fix for this is on Demo in release v1.1.0.41.

livebe01 avatar Nov 03 '25 21:11 livebe01

As of 11/12/25, the fix for this is on Prod in release v1.1.0.41.

livebe01 avatar Nov 17 '25 18:11 livebe01