modelmesh-runtime-adapter icon indicating copy to clipboard operation
modelmesh-runtime-adapter copied to clipboard

feat: Add ModelStreamInfer to Triton MethodInfos

Open Legion2 opened this issue 1 year ago • 1 comments

Motivation

Fix #80

Modifications

Add ModelStreamInfer to triton MethodInfos

Result

inference.GRPCInferenceService/ModelStreamInfer gRPC requests can be send to triton, which enable the use of triton backends and models which require streaming.

Legion2 avatar Jan 21 '24 15:01 Legion2

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Legion2 Once this PR has been reviewed and has the lgtm label, please assign ckadner for approval by writing /assign @ckadner in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

oss-prow-bot[bot] avatar Jan 21 '24 15:01 oss-prow-bot[bot]