fade_away

branch main，commit id：66ef1df492f7bc9c8eeb01d7e14db01838e3f0bd ``` model=/data/vicuna-13b/vicuna-13b-v1.5/ tp=2 python convert_checkpoint.py --model_dir ${model} \ --output_dir ./tllm_checkpoint_2gpu_fp16 \ --dtype float16 --tp_size ${tp} trtllm-build --checkpoint_dir ./tllm_checkpoint_2gpu_fp16 \ --output_dir ./tmp/llama/13B/trt_engines/fp16/2-gpu \ --gemm_plugin float16 \ --use_fused_mlp \...

> You can check the example version [here](https://github.com/NVIDIA/TensorRT-LLM/blob/main/tensorrt_llm/version.py) and compare it with the version of tensorrt_llm package. Hi I checked it, it's `0.9.0.dev2024030500`

> > And I also want to know what is the purpose for materialize_type? Is there any keywords to learn about this programming paradigm? > > Basically, `materialize_type` will create...

> wow，thank you! I suggest that you put this message in README. It's a very good feature.

> Do you have more profile data by nsight-compute? Which can be a good guide for perf debugging. > > BTW, have you done any autotune your layer norm triton...

Hey, I can try to answer the question, seems it's here :