serve
serve copied to clipboard
Include missing model configuration values in describe model API response
Description
Since configuration options can be set at various levels, i.e command line, TorchServe configuration file, model configuration file etc.., it is useful to include what these configuration values finally resolve to in the describe model API response.
This PR includes some of the missing configuration values in the describe model API response.
Fixes #3037
Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
Feature/Issue validation/testing
- [ ] CI
- [ ] Manual test
$ torchserve --ncs --start --model-store ./model_store
$ curl -X POST "http://127.0.0.1:8081/models?url=resnet-18.mar&initial_workers=1"
$ curl "http://127.0.0.1:8081/models/resnet-18"
[
{
"modelName": "resnet-18",
"modelVersion": "1.0",
"modelUrl": "resnet-18",
"runtime": "python",
"minWorkers": 1,
"maxWorkers": 1,
"batchSize": 1,
"maxBatchDelay": 100,
"responseTimeout": 120,
"maxRetryTimeoutInSec": 300,
"clientTimeoutInMills": 0,
"parallelType": "pp",
"parallelLevel": 1,
"deviceType": "gpu",
"deviceIds": [
0
],
"continuousBatching": false,
"useJobTicket": false,
"useVenv": false,
"stateful": false,
"sequenceMaxIdleMSec": 0,
"maxNumSequence": 0,
"maxSequenceJobQueueSize": 0,
"loadedAtStartup": false,
"workers": [
{
"id": "29500",
"startTime": "2024-04-30T22:58:40.635Z",
"status": "READY",
"memoryUsage": 0,
"pid": 5719,
"gpu": true,
"gpuUsage": "gpuId::0 utilization.gpu [%]::0 % utilization.memory [%]::0 % memory.used [MiB]::320 MiB"
}
],
"jobQueueStatus": {
"remainingCapacity": 100,
"pendingRequests": 0
}
}
]