BentoML icon indicating copy to clipboard operation
BentoML copied to clipboard

feature(frameworks): bentoml.onnx runner accept kwargs

Open larme opened this issue 2 years ago • 5 comments

What does this PR address?

Sometimes it's more nature to call onnx model with keyword arguments. For example, bert's tokenizer will output a dictionary, which the original model will call with model(**input). Our runner should simulate this behavior

Fixes #(issue)

Before submitting:

larme avatar Feb 14 '23 00:02 larme

Codecov Report

Merging #3561 (607ec46) into main (afe9660) will not change coverage. The diff coverage is 0.00%.

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #3561   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files        154     154           
  Lines      12620   12632   +12     
=====================================
- Misses     12620   12632   +12     
Impacted Files Coverage Δ
src/bentoml/_internal/frameworks/onnx.py 0.00% <0.00%> (ø)

codecov[bot] avatar Feb 14 '23 00:02 codecov[bot]

cc @larme, I ran the test locally, it passed, but it still didn't run on CI. can you also fix it as well?

aarnphm avatar Feb 14 '23 08:02 aarnphm

@aarnphm Now the tests run on CI. The problem is that the new bert tests will missing the pytorch dependencies (I thought installing transformers will install torch by default, I was wrong).

larme avatar Feb 21 '23 20:02 larme

@larme @aarnphm what's the status of this one?

sauyon avatar Aug 30 '23 00:08 sauyon

This has been on this list for a while now @larme we should circle back and get this one merge

aarnphm avatar Aug 30 '23 01:08 aarnphm