sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Create deployment folder for transformers integration

Open dbogunowicz opened this issue 3 years ago • 1 comments

Testing: First I set up the sparsezoo model (picked a stub and downloaded it). I remove deployment directory:

(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ ls
deployment  model.md  model.onnx  recipe  sample_inputs.tar.gz  sample_outputs.tar.gz  training

(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ rm -rf deployment/

(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ ls training/
config.json  pytorch_model.bin  recipe.yaml  special_tokens_map.json  tokenizer.json  tokenizer_config.json  trainer_state.json  training_args.bin  vocab.txt

Secondly, I run

sparseml.transformers.export_onnx  
--task
qa
--model_path
/home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/training

and get the output:

...
INFO:sparseml.pytorch.sparsification.quantization.quantize_qat_export:Converted 12 quantizable MatMul ops to QLinearMatMul
2022-08-11 10:16:04 sparseml.pytorch.sparsification.quantization.quantize_qat_export INFO     Converted 37 quantizable MatMul ops with weight and bias to MatMulInteger and Add
INFO:sparseml.pytorch.sparsification.quantization.quantize_qat_export:Converted 37 quantizable MatMul ops with weight and bias to MatMulInteger and Add
2022-08-11 10:16:05 __main__     INFO     ONNX exported to /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/training/model.onnx
fb6441a2ec36/training/model.onnx
2022-08-11 10:16:05 __main__     INFO     Model exported to: /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/training/model.onnx
2022-08-11 10:16:05 __main__     INFO     Saved tokenizer.json in the deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment/tokenizer.json
2022-08-11 10:16:05 __main__     INFO     Saved config.json in the deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment/config.json
2022-08-11 10:16:05 __main__     INFO     Saved model.onnx in the deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment/model.onnx
2022-08-11 10:16:05 __main__     INFO     Created deployment folder at /home/damian/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36/deployment

Finally:

(sparsezoo_venv) damian@lambdaquad:~/.cache/sparsezoo/9768075d-2cf0-4bc7-98c3-fb6441a2ec36$ ls deployment/
config.json  model.onnx  tokenizer.json

dbogunowicz avatar Aug 11 '22 14:08 dbogunowicz

@KSGulin assigned for review

github-actions[bot] avatar Aug 11 '22 14:08 github-actions[bot]

LGTM pending investigation of failing integration tests

bfineran avatar Aug 18 '22 01:08 bfineran