google
`xnn_weights_cache_provider` look_up doesn't work?
I've recently been experimenting with XNNPACK's weight cache to reduce load time by caching packed weights and also reduce memory pressure for repeated weights across the same kernels.
I was experiementing with fully-connected operator and found that the weight cache was never being hit. I noticed that when using the apis to create the xnn_weights_cache_t we set the look up function to be xnn_internal_weights_cache_look_up:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/runtime.c#L148
looking at this function, it looks like a placeholder function which would always return XNN_CACHE_NOT_FOUND:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/cache.c#L491-L496
Now when I'm using the weights cache to create a runtime_t with only a fully connected operator, in the flow of creating the fully-connected operator, we look up the cache to see if the weights have been packed before, using xnn_weights_cache_look_up:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/operators/fully-connected-nc.c#L154-L157
However this just uses the the placeholder function above, returning XNN_CACHE_NOT_FOUND:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/cache.c#L530-L534
As a result, every look up would then fall to XNN_CACHE_NOT_FOUND, in which weights have to be repacked, and memory has to be allocated for the newly packed weights:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/operators/fully-connected-nc.c#L159-L179
Am I looking at this incorrectly? Or is this a feature that is still a wip? Or is this a bug that is meant to be fixed in the future?
@mcr229 - Is it possible to not use the default constructor i.e. xnn_create_weights_cache_with_size and write a custom constructor which populates struct xnn_weights_cache_provider with your methods?
@digantdesai hmm I'm wondering if this suggests that XNNPACK doesn't offer a default operator for looking up weights then?
Hi, is this still an issue? Do you want to discuss during the next meeting?
