serve icon indicating copy to clipboard operation
serve copied to clipboard

Include missing model configuration values in describe model API response

Open namannandan opened this issue 9 months ago • 0 comments

Description

Since configuration options can be set at various levels, i.e command line, TorchServe configuration file, model configuration file etc.., it is useful to include what these configuration values finally resolve to in the describe model API response.

This PR includes some of the missing configuration values in the describe model API response.

Fixes #3037

Type of change

  • [x] Bug fix (non-breaking change which fixes an issue)

Feature/Issue validation/testing

  • [ ] CI
  • [ ] Manual test
$ torchserve --ncs --start --model-store ./model_store
$ curl -X POST "http://127.0.0.1:8081/models?url=resnet-18.mar&initial_workers=1"
$ curl "http://127.0.0.1:8081/models/resnet-18"
[                                                                                                                                                                                   
  {                                                                                                                                                                                 
    "modelName": "resnet-18",                                                                                                                                                       
    "modelVersion": "1.0",                                                                                                                                                          
    "modelUrl": "resnet-18",                                                                                                                                                        
    "runtime": "python",                                                                                                                                                            
    "minWorkers": 1,
    "maxWorkers": 1,
    "batchSize": 1,
    "maxBatchDelay": 100,
    "responseTimeout": 120,
    "maxRetryTimeoutInSec": 300,
    "clientTimeoutInMills": 0,
    "parallelType": "pp",
    "parallelLevel": 1,
    "deviceType": "gpu",
    "deviceIds": [
      0
    ],
    "continuousBatching": false,
    "useJobTicket": false,
    "useVenv": false,
    "stateful": false,
    "sequenceMaxIdleMSec": 0,
    "maxNumSequence": 0,
    "maxSequenceJobQueueSize": 0,
    "loadedAtStartup": false,
    "workers": [
      {
        "id": "29500",
        "startTime": "2024-04-30T22:58:40.635Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 5719,
        "gpu": true,
        "gpuUsage": "gpuId::0 utilization.gpu [%]::0 % utilization.memory [%]::0 % memory.used [MiB]::320 MiB"
      }
    ],
    "jobQueueStatus": {
      "remainingCapacity": 100,
      "pendingRequests": 0
    }
  }
]

namannandan avatar Apr 30 '24 20:04 namannandan