请问，gpt2 转换成lightseq后，支持传入token_type_ids吗？

[yuanhuachao]

huggingface gpt2转换成 lightseq gpt.pb后，传入token_type_ids后，会报

tensorrtserver.api.InferenceServerException: [ 0] unknown input 'token_type_ids' for 'gpt_generation' 

在训练时，是传入token_type_ids的，没有模型结构改变。

[hexisyztem]

Sorry, I missed your message last week. If you are using an older version of tensorrtserver, the inputs are named "inputs_ids", and the logic is determined in the underlying code: https://github.com/bytedance/lightseq/blob/master/lightseq/inference/server/gpt_generate_server .cc.cu#L255

[hexisyztem]

Sorry, I just misunderstood what you meant. Currently we do not support token_type_ids.

[yuanhuachao]

嗯，好的，感谢回复。因为huggingface的GPT2的token_type_embedding的实现是复用了token_embedding，网络结构上和不加token_type_ids没有区别，以为lightseq GPT2能支持的。🙏