server icon indicating copy to clipboard operation
server copied to clipboard

[python backend] Add class_count argument for inference requests with BLS scripting

Open Rusteam opened this issue 3 years ago • 3 comments

Is your feature request related to a problem? Please describe. I want to write a BLS with python backend that sends a request to a classifier with that has label_filename enabled in outputs. It's possible to do that with python tritonclient as InferRequestedOutput(name, class_count=top_k) that returns class names as well. It would be very handy to have the same functionality for pb_utils.InferenceRequest.

Describe the solution you'd like I want to send a request from python backend to a classifier and get response that includes class probs along with class labels.

request = pb_utils.InferenceRequest(
            model_name=model_name,
            requested_output_names=pbutils.InferRequestedOutput(name, class_count=3), # change
            inputs=[pb_utils.Tensor("image", images)]
        )
resp = request.exec()

output = pb_utils.get_output_tensor_by_name(resp, name)
print(output)

### prints
### [[b'3.489491:79:100078', b'3.372420:255:100270', b'3.222686:78:100079']]

Describe alternatives you've considered Duplicate labels.txt from the classifier to the python backend and parse values manually, but that includes duplication and extra code.

Rusteam avatar Jul 14 '22 16:07 Rusteam

Thanks for submitting your request. @Tabrizian Is this a common ask? If so, I can file an enhancement ticket for this.

krishung5 avatar Jul 15 '22 00:07 krishung5

It is the first time but I think it is a reasonable ask. Please file a ticket for it. Thanks.

Tabrizian avatar Jul 15 '22 15:07 Tabrizian

Filed ticket 3970.

krishung5 avatar Jul 15 '22 18:07 krishung5