Junhong Liu

Results 19 comments of Junhong Liu
trafficstars

BERT Model is Inaccurate

@Lzhang-hub Do you still have the question? If not, we will close it soon.

[Feature Request] Support for Constrained Decoding (such as generating Json formatted output)

cc @AdamzNV @ncomly-nvidia @laikhtewari for vis.

Encounter CUDA error when increasing the length of input_ids

@1649759610 Do you still have the question? If not, we will close it soon.

calibration dataset

@Hukongtao Do you still have the problem? If not, we will close it soon.

Support use_cache=False

@Hukongtao Do you still have the question? If not, we will close it soon.

Conversion of "hf_lora_convert.py" does not account for "lora_alpha"

@TheCodeWrangler Do you still have the question? If not, we will close it soon.

Does TensorRT-LLM support passing input_embeds directly？

@Hukongtao If you have no further questions, we will close it in a week.

failed to use TensorRT-LLM/examples/apps/fastapi_server.py

@AGI-player could you please try to use the latest main branch?

Qwen2-72B-Instruct-GPTQ-Int4 Conversion Success, Run Failure

@linchpinlin If you have no further question, we will close this issue in one week.

Finding protobuf files while benchmarking TensorRT-LLM

@KuntaiDu If you have no further question, we will close it in a week.