ggml-org
CANN: Use smart pointers to manage ACL objects
Previously, ACL objects were managed via manual destruction, which led to multiple memory-leak issues during runtime. This patch replaces manual memory management with smart pointers so that ACL objects are properly released and ownership is clearly defined.
Note that the ownership of an ACL object belongs to the function that creates it. Other internal functions should operate on these ACL objects using raw pointers to avoid unintended ownership transfers.
Additionally, since aclTensorList automatically frees its contained aclTensor objects, any aclTensor added to a tensor list must release ownership to avoid double free operations.
This PR also removes the asynchronous task submission mechanism. Due to changes in recent CANN versions, tiling time has significantly decreased. Even with a dual-thread submission model, the dispatch overhead still falls on the critical path, making async submission less beneficial. Moreover, aclGraph support provides a much better path to reducing operator dispatch latency.
Make sure to read the contributing guidelines before submitting a PR
Test case passwd：
2701/2701 tests passed Backend CANN0: OK Backend 2/3: CANN1 Skipping Backend 3/3: CPU Skipping 3/3 backends
Additionally, I think we still need to verify that there are currently no memory leaks.
Additionally, I think we still need to verify that there are currently no memory leaks.
I ran Valgrind on llama-server and didn’t find any leaks. There were a few false positives, though, coming from inside CANN.
在310P3上 编译报错 /llama.cpp/ggml/src/ggml-cann/aclnn_ops.cpp:2628:24: 错误：cannot convert ‘acl_tensor_ptr’ {aka ‘std::unique_ptr<aclTensor, acl_deleter<aclTensor, aclDestroyTensor> >’} to ‘aclTensor*’
在310P3上 编译报错 /llama.cpp/ggml/src/ggml-cann/aclnn_ops.cpp:2628:24: 错误：cannot convert ‘acl_tensor_ptr’ {aka ‘std::unique_ptr<aclTensor, acl_deleter<aclTensor, aclDestroyTensor> >’} to ‘aclTensor*’
My bad. Forget to run testcase in 310p. see #17347