classy-classification
classy-classification copied to clipboard
davidberenstein1957
The current version of package is unstable and exceptions occur
Hello,
I try to use the package with ONNX runtime, but the current version works very unstable and doesn't work in most cases. Also it seems fast-sentence-transformers package is outdated.
I try to run examples from Readme and I get errors.
OS: Ubuntu 20.04
Installing:
pip install classy-classification[onnx]
SpaCy embeddings. It does not work.
Error:
...
File ~/tmp/classy/lib/python3.11/site-packages/classy_classification/classifiers/classy_spacy.py:101, in ClassySpacyInternal.get_embeddings(self, docs)
99 elif doc.has_extension("trf_data"):
100 print(doc)
--> 101 embeddings.append(doc._.trf_data.model_output.pooler_output[0])
102 else:
103 warnings.warn(
104 f"None of the words in the text `{str(doc)}` have vectors. Returning zeros.", stacklevel=1
105 )
AttributeError: 'DocTransformerOutput' object has no attribute 'model_output'
Sentence-transfomer embeddings. It works.
Hugginface zero-shot classifiers. It does not work.
Error:
...
File ~/tmp/classy/lib/python3.11/site-packages/optimum/exporters/onnx/__main__.py:356, in main_export(model_name_or_path, output, task, opset, device, fp16, optimize, monolith, no_post_process, framework, atol, cache_dir, trust_remote_code, pad_token_id, subfolder, revision, force_download, local_files_only, use_auth_token, for_ort, do_validation, model_kwargs, custom_onnx_configs, fn_get_submodels, use_subprocess, _variant, library_name, legacy, **kwargs_shapes)
354 autodetected_message = ""
355 model_tasks = TasksManager.get_supported_tasks_for_model_type(model_type, exporter="onnx")
--> 356 raise ValueError(
357 f"Asked to export a {model_type} model for the task {task}{autodetected_message}, but the Optimum ONNX exporter only supports the tasks {', '.join(model_tasks.keys())} for {model_type}. Please use a supported task. Please open an issue at https://github.com/huggingface/optimum/issues if you would like the task {task} to be supported in the ONNX export for {model_type}."
358 )
360 # TODO: Fix in Transformers so that SdpaAttention class can be exported to ONNX. `attn_implementation` is introduced in Transformers 4.36.
361 if model_type in SDPA_ARCHS_ONNX_EXPORT_NOT_SUPPORTED and _transformers_version >= version.parse("4.35.99"):
ValueError: Asked to export a bart model for the task text-classification, but the Optimum ONNX exporter only supports the tasks feature-extraction, feature-extraction-with-past, text-generation, text-generation-with-past, text2text-generation, text2text-generation-with-past for bart. Please use a supported task. Please open an issue at https://github.com/huggingface/optimum/issues if you would like the task text-classification to be supported in the ONNX export for bart.
Standalone usage without spaCy
It works with the default model: torch/sentence_transformers/sentence-transformers_paraphrase-multilingual-MiniLM-L12-v2/quantized_true.onnx
The following code does not work if the model is not yet downloaded:
# overwrite [embedding model](https://www.sbert.net/docs/pretrained_models.html)
classifier.set_embedding_model(model="paraphrase-MiniLM-L3-v2")
Error:
...
File ~/tmp/classy/lib/python3.11/site-packages/fast_sentence_transformers/FastSentenceTransformer.py:203, in FastSentenceTransformer.model2onnx(self)
201 print(f"Model found at: {self.export_model_name}")
202 else:
--> 203 self.onnx(self.model_path, "default", self.export_model_name, quantize=self.quantize)
204 print(f"Model exported at: {self.export_model_name}")
File ~/tmp/classy/lib/python3.11/site-packages/fast_sentence_transformers/txtai/pipeline/train/hfonnx.py:124, in HFOnnx.__call__(self, path, task, output, quantize, opset)
121 if not ONNX_RUNTIME:
122 raise ImportError('onnxruntime is not available - install "pipeline" extra to enable')
--> 124 output = self.quantization(output)
126 if isinstance(output, BytesIO):
127 # Reset stream and return bytes
128 output.seek(0)
File ~/tmp/classy/lib/python3.11/site-packages/fast_sentence_transformers/txtai/pipeline/train/hfonnx.py:162, in HFOnnx.quantization(self, output)
160 output = Path(output)
161 # Quantize optimized model
--> 162 quantize_dynamic(output, output, optimize_model=False)
164 # Read file back to bytes if temp file was created
165 if temp:
TypeError: quantize_dynamic() got an unexpected keyword argument 'optimize_model'
The second run works without errors.
pip list:
Package Version
-------------------------- ------------
accelerate 0.26.1
aiohttp 3.9.1
aiosignal 1.3.1
annotated-types 0.6.0
asttokens 2.4.1
attrs 23.2.0
blis 0.7.11
catalogue 2.0.10
certifi 2023.11.17
charset-normalizer 3.3.2
classy-classification 0.6.7
click 8.1.7
cloudpathlib 0.16.0
coloredlogs 15.0.1
confection 0.1.4
curated-tokenizers 0.0.9
curated-transformers 0.1.1
cymem 2.0.8
datasets 2.16.1
decorator 5.1.1
dill 0.3.7
en-core-web-trf 3.7.3
evaluate 0.4.1
executing 2.0.1
fast-sentence-transformers 0.4.1
filelock 3.13.1
flatbuffers 23.5.26
frozenlist 1.4.1
fsspec 2023.10.0
huggingface-hub 0.20.3
humanfriendly 10.0
idna 3.6
ipython 8.20.0
jedi 0.19.1
Jinja2 3.1.3
joblib 1.3.2
langcodes 3.3.0
MarkupSafe 2.1.4
matplotlib-inline 0.1.6
mpmath 1.3.0
multidict 6.0.4
multiprocess 0.70.15
murmurhash 1.0.10
networkx 3.2.1
nltk 3.8.1
numpy 1.26.3
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
onnx 1.15.0
onnxruntime 1.16.3
optimum 1.16.2
packaging 23.2
pandas 1.5.3
parso 0.8.3
pexpect 4.9.0
pillow 10.2.0
pip 23.3.2
preshed 3.0.9
prompt-toolkit 3.0.43
protobuf 4.25.2
psutil 5.9.8
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 15.0.0
pyarrow-hotfix 0.6
pydantic 2.5.3
pydantic_core 2.14.6
Pygments 2.17.2
python-dateutil 2.8.2
pytz 2023.3.post1
PyYAML 6.0.1
regex 2023.12.25
requests 2.31.0
responses 0.18.0
safetensors 0.4.2
scikit-learn 1.4.0
scipy 1.12.0
sentence-transformers 2.2.2
sentencepiece 0.1.99
setuptools 69.0.3
six 1.16.0
smart-open 6.4.0
spacy 3.7.2
spacy-alignments 0.9.1
spacy-curated-transformers 0.2.1
spacy-legacy 3.0.12
spacy-loggers 1.0.5
spacy-transformers 1.3.4
srsly 2.4.8
stack-data 0.6.3
sympy 1.12
thinc 8.2.2
threadpoolctl 3.2.0
tokenizers 0.15.1
torch 2.1.2
torchvision 0.16.2
tqdm 4.66.1
traitlets 5.14.1
transformers 4.36.2
triton 2.1.0
typer 0.9.0
typing_extensions 4.9.0
urllib3 2.1.0
wasabi 1.1.2
wcwidth 0.2.13
weasel 0.3.4
wheel 0.42.0
xxhash 3.4.1
yarl 1.9.4
Will the package run more stable without ONNX? Maybe we need to specify the critical dependency versions more strictly? Right now it's working very erratically and it's not clear at what point things might break down.
Hi @espdev , what do you propose for specifying versioning ? PRs are welcome :)
Hi @davidberenstein1957,
It seems the dependency versions currently specified in pyproject.toml are not fully compatible with the package code. I think we need to specify the version ranges of the major dependencies with which the package will definitely work without errors (I hope semver will work). spaCy, sentence-transformers, transformers, onnx/onnxruntime. Also it seems fast-sentence-transformers package is outdated for the current version of sentence-transformers package.