Chevolier

Does this program supports tensorboard?

2

Does this program supports tensorboard? Could not find any logs of tensorbard.

Last token repeat after adding end_id for Baichuan2-13B-Chat

2

Build the engine using the following code (TensorRT-LLM version v0.7.1) for Baichuan2-13B-Chat model. python build.py --model_version v2_13b \ --model_dir /code/tensorrt_llm/checkpoint/hf_bf16 \ --dtype float16 \ --use_gemm_plugin float16 \ --use_gpt_attention_plugin float16 \...

torchrec Build inference library and example server failure

2

I followed the steps in https://github.com/pytorch/torchrec/tree/main/torchrec/inference to test inference. But in 4. Build inference library and example server, the Build server and C++ protobufs failed. In particular, after I input...