SaltyEggXman comments

Results 9 comments of


                                            SaltyEggXman

trafficstars

gptSessionBenchmark Failed Because of " Assertion failed: d == a + length " with 0.7.1 Release in tritonserver:23.12-trtllm-python-py3 Image

Yes. The previous version works. This seems to be a new issue in the new 0.7.1 release.

gptSessionBenchmark Failed Because of " Assertion failed: d == a + length " with 0.7.1 Release in tritonserver:23.12-trtllm-python-py3 Image

@wjj19950828 Which version of the tensorrtllm folder you used for the replacement? I am using the folder in `23.11-trtllm-python-py` to replace the one in `23.12-trtllm-python-py`. It has the same error....

[FP8 Post-Training Quantization] "use_fp8_context_fmha" Not Supported As Description

Yes, tried in 0.9.0, it is supported now. What is the relationship between `--use_fp8_context_fmha` and `--context_fmha enable`. If I use `--use_fp8_context_fmha`, do I still need --context_fmha enable`? In general, is...

Llama-2-7b - Tensor Dimension Mismatch Error for AWQ Engine Build for 4 GPUs

Do I need to rebuild the docker image for the main branch? I noticed that there were some files updated under docker folder. From the notes, it seems there are...

Llama-2-7b - Tensor Dimension Mismatch Error for AWQ Engine Build for 4 GPUs

I reused the docker image built for 0.5.0 branch and generated a new container with the following command: `REPOSITORY TAG IMAGE ID CREATED SIZE tensorrt_llm/release latest-root 5e43c4749c11 41 hours ago...

Llama-2-7b - Tensor Dimension Mismatch Error for AWQ Engine Build for 4 GPUs

Tried with the latest main branch code and rebuilt the docker image. TP size = 4 still gave me the following errors: ``` [12/12/2023-00:21:08] [TRT-LLM] [E] Current weight shape is...

Llama-2-7b - Tensor Dimension Mismatch Error for AWQ Engine Build for 4 GPUs

In 0.5 release, the generated file is .pt. Why in this version, it is generated as .npz file?

make -C docker run LOCAL_USER=1 FAILED

I have exact the same issue. Any update on this?

make -C docker run LOCAL_USER=1 FAILED

It seems you have to run ` make -C docker release_run LOCAL_USER=1` instead of `make -C docker run LOCAL_USER=1`. Otherwise, it will look for the devel version of the image...