1826133674
1826133674
I have the same problem too. Have you understood it well? Would you mind tell me your idea to me ? Thank you very much!
相关的环境变量 declare -x FLAGS_allocator_strategy="naive_best_fit" declare -x FLAGS_cache_inference_while_scope="1" declare -x FLAGS_control_flow_use_new_executor="1" declare -x FLAGS_cublaslt_exhaustive_search_times="10" declare -x FLAGS_fraction_of_gpu_memory_to_use="0.92" declare -x FLAGS_new_executor_serial_run="1" declare -x FLAGS_use_autotune="1"
@lizexu123 怎么确认使用的是原生gpu还是trt推理？
@lizexu123 没有出现 --- Running analysis [ir_graph_build_pass] I0207 15:37:05.177603 2824 executor.cc:187] Old Executor is Running. --- Running analysis [ir_analysis_pass] --- Running IR pass [map_op_to_another_pass] --- Running IR pass [identity_op_clean_pass] I0207 15:37:09.198072...
我的所有版本是这样子的，我不太清楚是不是符合你说的这个，我有个问题是，同样的量化权重，动态图就可以推理，静态图则不行，这跟paddle的实现有关吗？这两个推理最终调用的paddle计算函数不一样吗？ paddle-bfloat 0.1.7 paddle2onnx 1.1.0 paddlefsl 1.1.0 paddlenlp 2.7.0.post0 paddlenlp-ops 0.0.0 paddlepaddle-gpu 2.6.0.post112
@chen-zhiyu 您好，完成tiny-llama的convert后，我修改了tokenizer.model的名字后，可以成功加载但是，推理不出正确的结果。请问您有遇到类似的情况吗？ 我的环境： paddle-bfloat 0.1.7 paddle2onnx 1.1.0 paddlefsl 1.1.0 paddlenlp 2.7.0.post0 paddlepaddle-gpu 2.6.0.post112 转换代码 from paddlenlp.transformers import AutoModelForCausalLM AutoModelForCausalLM.from_pretrained("./TinyLlama-1.1B-Chat-v1.0", convert_from_torch=True, dtype="float16") 推理代码 from paddlenlp.transformers import AutoModelForCausalLM, AutoTokenizer paddle.set_device("gpu:3") print("paddle.device",paddle.get_device()) model...
I meet this error in aimet_torch 1.31.0. Any suggestions to solve this problem? Name: aimet-torch Version: torch-gpu-1.31.0 Summary: AIMET torch Package Home-page: https://github.com/quic/aimet/releases/download/torch-gpu_1.31.0 Author: Qualcomm Innovation Center, Inc. Author-email: [email protected]...
> you need to change the onnx simplifier to onnx_utils.simplify_onnx_model = False Thank you for your reply！I'v solve it according your tips! By the way, the code is in file...
> HI, even i had the same issue, can you help me in resolving the issue. @sandeep1404 Did you sovle the problem?
> HI, even i had the same issue, can you help me in resolving the issue. You can download the patch from this link https://github.com/quic/aimet/blob/develop/Jenkins/opencv_320_python38.patch