bug: RuntimeError on tensor size mismatch despite setting truncation in transformer pipeline when saving model with BentoML

[FahriBilici]

Describe the bug

I am encountering a RuntimeError about tensor size mismatch when attempting to save a transformer model with BentoML, despite having set truncation in the transformer pipeline. The error message is: RuntimeError: The expanded size of the tensor (713) must match the existing size (514) at non-singleton dimension 1.

To reproduce

first creating pipeline pipe = transformers.pipeline(task, model=model, truncation=True, max_length=512) saving model to bentoml bentoml.transformers.save_model( task, pipe, metadata=dict(model_name=model), ) Then send very long text(more than 512 token) for reproduce

Expected behavior

model should truncate and do prediction.

Environment

bentoml: 1.1.8 python: 3.10.13 platform macos (m2)