Quang-elec44

@cg123 I set `clone_tensors=True` in `MergeOptions` class and still got the same error 

@Ar57m Oh my bad. Thanks for your notice.

@cg123 @Ar57m It worked. Thanks for your help !!!

Hi @mrwyattii , may I ask how to keep the restful server alive ? Here is my script ```python import mii mii_configs = { "tensor_parallel": 2, "dtype": "fp16", "enable_restful_api": True,...

@satpalsr Not really! My first attempt with ```--num_gpus >3``` failed without any previous run

@udhavsethi According to the tutorial page, at [this part](https://www.deepspeed.ai/tutorials/inference-tutorial/#end-to-end-gpt-neo-27b-inference), you can get the result from rank 0. About model parallelism, in my experience, it didn't work as I expected. It...

Hi everyone, I tried to export a MBartForConditionalGeneration and all exporting processes worked well. However, when I use `onnxruntime` to load the model, it raises the following error: `onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError]...

@tianleiwu Here is my script ``` import os import onnx from onnx import TensorProto, helper from utils import export_helper def make_dim_proto_numeric(model, config): """Make dim_proto numeric. Args: model (BartForConditionalGeneration): Bart model....

@tianleiwu It's weird that the beam search input is in the range of 5 and 10, and btw, why there are some empty string inputs?

@tianleiwu Oh I see, some inputs in class `GenerationConfig` are not available exist in the 5-10 inputs. In my script, I have a additional inputs (e.g `emotion_mask`), which is an...