Qinqing Liu

Tried `python3 run.py compare BART --variant facebook/bart-base --working-dir temp` also get error: Collecting Data for onnxrt Traceback (most recent call last): File "run.py", line 297, in main() File "run.py", line...

It works with the use_cache=False. Is there any effects without cache?

Thank you for the info! It seems a large batch size is also not supported yet. Could you please confirm?

I create 32 duplicates for input and specified the batch size, and it should be passed through the profile. However I get the error. `inputs = tokenizer(["translate English to German:...

The batch size assignments are before the tenorRT engine. I am following https://github.com/NVIDIA/TensorRT/blob/main/demo/HuggingFace/notebooks/t5.ipynb, and the batch size was assigned at the very beginning of tensorRT section. The code below are...

I see. I was using the old engine built with batch_size=1. Another modification: for decode profile, we should have **hidden_dim** = BARTModelTRTConfig.ENCODER_HIDDEN_SIZE[model_name] decoder_profile.add( "encoder_hidden_states", min=(batch_size, 1, **hidden_dim**), opt=(batch_size, max_sequence_length //...

Is it a command or convenient way to set up the engine for a local checkpoint of fine-tuned bart model, or a customized bart model?

> Yes, the `as_trt_engine` lines are where the engines got really built. Did you see some log in the notebook like TRT is building the engine (and usually this engine...

Thank you for your sharing! Do you have any idea what would be appropriate to set up keras_utils in kwargs? Thank you for your help!

By setting: backend=keras.backend, layers = keras.layers, models = keras.models, utils = keras.utils as kwargs, problem solved. Thank you.