tvm icon indicating copy to clipboard operation
tvm copied to clipboard

[Runtime] Add runtime.contrib.set_custom_cpu_allocator to use custom memalign() and free() functions

Open jsheng-jian opened this issue 2 years ago • 3 comments

This PR adds a new function runtime.contrib.set_custom_cpu_allocator to the registry which can be used to override the memalign() and free() functions used by CPUDeviceAPI. Upstream original PR by @trevor-m .

cc @areusch

jsheng-jian avatar Jun 05 '22 01:06 jsheng-jian

@jsheng-jian thanks for the PR! i think this maybe deserves a broader look by the community as it could potentially impact performance. just curious, did you consider a compile-time workaround here as well? would that work for your use case, or does it need to always be available?

areusch avatar Jun 07 '22 18:06 areusch

cc @tqchen @csullivan @junrushao1994 @jwfromm

areusch avatar Jun 07 '22 18:06 areusch

Having an extra null pointer check is not going to be significantly affect performance, but I would be curious on the implication of user-facing API.

There could be some compile-time alternatives, for example, using macros to control which alloc/free methods to use. Do you think it's viable in your usecase?

Thanks a lot!

junrushao avatar Aug 02 '22 16:08 junrushao

@jsheng-jian could you address the question above?

areusch avatar Aug 11 '22 00:08 areusch