openvinotoolkit
[GPU] Weightless caching
Did you check accuracy? When I tested this PR with the resnet-18 static model, the outputs are different between non-caching and weightless-caching runs. Additionally, I temporarily commented out the below two lines for weightless cache blob loading: https://github.com/openvinotoolkit/openvino/blob/7cf05641b6b1b249904c96e02ac07ee384219bb4/src/plugins/intel_gpu/src/plugin/plugin.cpp#L311-L312
Did you check accuracy? When I tested this PR with the resnet-18 static model, the outputs are different between non-caching and weightless-caching runs. Additionally, I temporarily commented out the below two lines for weightless cache blob loading:
https://github.com/openvinotoolkit/openvino/blob/7cf05641b6b1b249904c96e02ac07ee384219bb4/src/plugins/intel_gpu/src/plugin/plugin.cpp#L311-L312
Hi, thanks for letting me know about issues with this topology! I checked accuracy only for Stable Diffusion v1.5 and Llama-3-8b. I'm aware that there may be mismatches in other topologies (see discussion here: https://github.com/openvinotoolkit/openvino/pull/25731#discussion_r1756722874).
I'm aware that this check prevents correct import, I'll push the fix soon.
Did you check accuracy? When I tested this PR with the resnet-18 static model, the outputs are different between non-caching and weightless-caching runs. Additionally, I temporarily commented out the below two lines for weightless cache blob loading:
https://github.com/openvinotoolkit/openvino/blob/7cf05641b6b1b249904c96e02ac07ee384219bb4/src/plugins/intel_gpu/src/plugin/plugin.cpp#L311-L312
Hi @e-ddykim, I get no mismatches with the current commit and a sample image. I tried to reproduce your issue with an old commit but got an exception throw instead of mismatches like you, so we have different setups. Could you please recheck to make sure it's also fixed on your setup?
Did you check accuracy? When I tested this PR with the resnet-18 static model, the outputs are different between non-caching and weightless-caching runs. Additionally, I temporarily commented out the below two lines for weightless cache blob loading: https://github.com/openvinotoolkit/openvino/blob/7cf05641b6b1b249904c96e02ac07ee384219bb4/src/plugins/intel_gpu/src/plugin/plugin.cpp#L311-L312
Hi @e-ddykim, I get no mismatches with the current commit and a sample image. I tried to reproduce your issue with an old commit but got an exception throw instead of mismatches like you, so we have different setups. Could you please recheck to make sure it's also fixed on your setup?
Now, I can get correct results from resnet-18 with this PR.