api-inference-community icon indicating copy to clipboard operation
api-inference-community copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Add support for neural compressor models

Open echarlaix opened this issue 2 years ago • 4 comments

echarlaix avatar May 08 '23 15:05 echarlaix

neural_compressor is going to become a framework right ? (library_tag in the README) .

Makes sense to me ! An other option could be to have optimum as library_tag, and have neural-compressor / openvino / onnx as tags, if needed we can also add the model loading logic to optimum (mentionning ONNX and OpenVINO as I would like to add their support as well)

What do you think would make more sense @Narsil @osanseviero ?

echarlaix avatar Jul 31 '23 13:07 echarlaix

What do you think would make more sense @Narsil @osanseviero ?

Both are fine to me. I don't necessarily know the scoping of this vs optimum and such.

Narsil avatar Jul 31 '23 14:07 Narsil

Also ignore the failing tests if they work locally for you. Somehow the CI has issues with docker signals.

Narsil avatar Jul 31 '23 15:07 Narsil

Perfect thanks @Narsil, I need to wait for updates from the Intel collaboration before merging, will change the PR status to draft temporarily

echarlaix avatar Aug 28 '23 12:08 echarlaix