sparseml
sparseml copied to clipboard
Create deployment folder for transformers integration
Testing: First I set up the sparsezoo model (picked a stub and downloaded it). I remove deployment directory:
(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ ls
deployment model.md model.onnx recipe sample_inputs.tar.gz sample_outputs.tar.gz training
(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ rm -rf deployment/
(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ ls training/
config.json pytorch_model.bin recipe.yaml special_tokens_map.json tokenizer.json tokenizer_config.json trainer_state.json training_args.bin vocab.txt
Secondly, I run
sparseml.transformers.export_onnx
--task
qa
--model_path
/home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/training
and get the output:
...
INFO:sparseml.pytorch.sparsification.quantization.quantize_qat_export:Converted 12 quantizable MatMul ops to QLinearMatMul
2022-08-11 10:16:04 sparseml.pytorch.sparsification.quantization.quantize_qat_export INFO Converted 37 quantizable MatMul ops with weight and bias to MatMulInteger and Add
INFO:sparseml.pytorch.sparsification.quantization.quantize_qat_export:Converted 37 quantizable MatMul ops with weight and bias to MatMulInteger and Add
2022-08-11 10:16:05 __main__ INFO ONNX exported to /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/training/model.onnx
fb6441a2ec36/training/model.onnx
2022-08-11 10:16:05 __main__ INFO Model exported to: /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/training/model.onnx
2022-08-11 10:16:05 __main__ INFO Saved tokenizer.json in the deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment/tokenizer.json
2022-08-11 10:16:05 __main__ INFO Saved config.json in the deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment/config.json
2022-08-11 10:16:05 __main__ INFO Saved model.onnx in the deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment/model.onnx
2022-08-11 10:16:05 __main__ INFO Created deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment
Finally:
(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ ls deployment/
config.json model.onnx tokenizer.json
@KSGulin assigned for review
LGTM pending investigation of failing integration tests