sagemaker-inference-toolkit icon indicating copy to clipboard operation
sagemaker-inference-toolkit copied to clipboard

Published 20 hours ago • verified publisher icon aws

Default output function encodes results to JSON and that seems to add to response latency.

Open vdantu opened this issue 3 years ago • 1 comments

Describe the bug By default the accept type in inference container seems to be application/json. The default encoder which converts results to JSON seems to add significantly to the response latencies. Is there a way to reduce the default response's latencies?

vdantu avatar Mar 30 '21 17:03 vdantu

up, did you find the solution?

Pawel842 avatar Nov 28 '23 18:11 Pawel842