[Bug] internvl 4B awq推理 Engine loop failed with error: module 'triton.language' has no attribute 'inline_asm_elementwise'

[Mrgengli]

Checklist

• [X] 1. I have searched related issues but cannot get the expected help.
• [X] 2. The bug has not been fixed in the latest version.
• [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

i have install triton-2.3.1,lmdeploy==0.6.1,but got the error below:

Reproduction

1

Environment

1

Error traceback

No response

[lvhan028]

Pls share the information after executing lmdeploy check_env

[Mrgengli]

thanks for your reply! this is the env below:

[AllentDan]

You were still using triton2.1.0. Please make sure the correct triton version was installed.

[Mrgengli]

@AllentDan which is the right version please?i have tried deferent version,but other problem is happend.

[AllentDan]

As lmdeploy check_env shows, you were using triton 2.1. Please install triton 2.3.0 correctly.

Or, you can use docker env. https://hub.docker.com/r/openmmlab/lmdeploy/tags

[Mrgengli]

@AllentDan I have correctly installed triton 2.3.0,but another error comes.

[grimoire]

Gcc 4.8.5 is not compatible with your triton environment.

We run a simple add kernel as the environment check before launching our engine.

https://github.com/InternLM/lmdeploy/blob/main/lmdeploy/pytorch/check_env/triton_custom_add.py

Please make sure that custom_add can be used in your environment.

[Mrgengli]

@grimoire I have successfully run triton_custom_add.py,but still got the same error.

[grimoire]

This Error comes from custom_add

https://github.com/InternLM/lmdeploy/blob/4492df812363d112aed9151a3ff7e26a654d36ea/lmdeploy/pytorch/check_env/init.py#L77-L81

Please double check custom_add and make sure the code above can be used.

[Mrgengli]

@grimoire I have run custom_add and got the error below.could you tell me how to solve it please?

[grimoire]

Gcc 4.8.5 is not compatible with your triton environment.

[Mrgengli]

@AllentDan hello,why I can not successfully pull the docker?

[Mrgengli]

@grimoire @AllentDan I have successfully run the awq model since I change the right version of gcc.Thanks for all your reply.