serve
serve copied to clipboard
Keep root dir for extra files when generating .mar file in model-archiver
Description
Please read our CONTRIBUTING.md prior to creating your first pull request.
Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes #1689
Type of change
- [x] New feature (non-breaking change which adds functionality)
- [ ] Documentation updated. see README in
examples/mar_extra_root/.
Feature/Issue validation/testing
Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced. Please also list any relevant details for your test configuration.
- [x] Test A
$ torch-model-archiver \
--model-name mnist \
--version 1.0 \
--model-file examples/image_classifier/mnist/mnist.py \
--serialized-file examples/image_classifier/mnist/mnist_cnn.pt \
--handler 'examples/mar_extra_root/mnist_handler_extra.py' \
--extra-files 'examples/mar_extra_root/my_lib1,examples/mar_extra_root/my_lib2'
ERROR - Maybe running with `--keep-extra-root` or rename the files
Traceback (most recent call last):
File "/opt/miniconda3/envs/torch/bin/torch-model-archiver", line 33, in <module>
sys.exit(load_entry_point('torch-model-archiver', 'console_scripts', 'torch-model-archiver')())
File "/home/joax/GitSpace/pytorch_serve/model-archiver/model_archiver/model_packaging.py", line 57, in generate_model_archive
package_model(args, manifest=manifest)
File "/home/joax/GitSpace/pytorch_serve/model-archiver/model_archiver/model_packaging.py", line 36, in package_model
model_path = ModelExportUtils.copy_artifacts(model_name, keep_extra_root=args.keep_extra_root,
File "/home/joax/GitSpace/pytorch_serve/model-archiver/model_archiver/model_packaging_utils.py", line 180, in copy_artifacts
ModelExportUtils.__copy_dir(file, model_path, keep_extra_root)
File "/home/joax/GitSpace/pytorch_serve/model-archiver/model_archiver/model_packaging_utils.py", line 146, in __copy_dir
shutil.copytree(src, dst, False, None)
File "/opt/miniconda3/envs/torch/lib/python3.9/shutil.py", line 565, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
File "/opt/miniconda3/envs/torch/lib/python3.9/shutil.py", line 466, in _copytree
os.makedirs(dst, exist_ok=dirs_exist_ok)
File "/opt/miniconda3/envs/torch/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/mnist/my_module'
- [x] Test B
$ torch-model-archiver \
--model-name mnist \
--version 1.0 \
--model-file examples/image_classifier/mnist/mnist.py \
--serialized-file examples/image_classifier/mnist/mnist_cnn.pt \
--handler 'examples/mar_extra_root/mnist_handler_extra.py' \
--extra-files 'examples/mar_extra_root/my_lib1,examples/mar_extra_root/my_lib2' \
--keep-extra-root
$ unzip -l mnist.mar
Archive: mnist.mar
Length Date Time Name
--------- ---------- ----- ----
1376 2022-06-19 02:06 mnist_handler_extra.py
4800893 2022-06-19 02:06 mnist_cnn.pt
757 2022-06-19 02:06 mnist.py
58 2022-06-19 01:32 my_lib2/my_file.py
59 2022-06-19 01:32 my_lib2/my_module/my_file1.py
58 2022-06-19 01:32 my_lib1/my_file.py
59 2022-06-19 01:32 my_lib1/my_module/my_file1.py
271 2022-06-19 02:06 MAR-INF/MANIFEST.json
--------- -------
4803531 8 files
Checklist:
- [x] Did you have fun?
- [x] Have you added tests that prove your fix is effective or that this feature works? Yes, see
examples/mar_extra_root/ - [x] Has code been commented, particularly in hard-to-understand areas?
- [x] Have you made corresponding changes to the documentation? Yes, see README in
examples/mar_extra_root/