Error while deploying with torchserve - Exception: Unable to create mar file

[saichandrapandraju]

Hi, I'm using 'BertNewsClassification' example to try torchserve with mlflow. But while deploying the model, I'm getting error like : Exception: Unable to create mar file. Here is the full trace of the error -

mlflow deployments create -t torchserve -m file:///C:/Users/saichandra.pandraju/Desktop/torch_models/ --name news_classification_test -C "VERSION=1.0" -C "MODEL_FILE=news_classifier.py" -C "HANDLER=news_classifier_handler.py" -C "EXPORT_PATH=C:\Users\saichandra.pandraju\Desktop\torch_serve\model_store\"
ERROR - Given export-path C:\Users\saichandra.pandraju\Desktop\torch_serve\model_store --model-file news_classifier.py --extra-files 'C:\Users\saichandra.pandraju\Desktop\torch_models\extra_files/class_mapping.json,C:\Users\saichandra.pandraju\Desktop\torch_models\extra_files/bert_base_uncased_vocab.txt' -r C:\Users\saichandra.pandraju\Desktop\torch_models\requirements.txt is not a directory. Point to a valid export-path directory.
Error when attempting to load and parse JSON cluster spec from file torch-model-archiver --force --model-name news_classification_test --version 1.0 --serialized-file C:\Users\saichandra.pandraju\Desktop\torch_models\data\model.pth --handler news_classifier_handler.py --export-path C:\Users\saichandra.pandraju\Desktop\torch_serve\model_store" --model-file news_classifier.py --extra-files 'C:\Users\saichandra.pandraju\Desktop\torch_models\extra_files/class_mapping.json,C:\Users\saichandra.pandraju\Desktop\torch_models\extra_files/bert_base_uncased_vocab.txt' -r C:\Users\saichandra.pandraju\Desktop\torch_models\requirements.txt
Traceback (most recent call last):
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\saichandra.pandraju\.conda\envs\mlflow\Scripts\mlflow.exe\__main__.py", line 7, in <module>
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\click\core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\click\core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\click\core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\click\core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\click\core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\mlflow\deployments\cli.py", line 132, in create_deployment
    deployment = client.create_deployment(name, model_uri, flavor, config=config_dict)
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\mlflow_torchserve\__init__.py", line 104, in create_deployment
    model_uri=model_uri,
  File "c:\users\saichandra.pandraju\.conda\envs\mlflow\lib\site-packages\mlflow_torchserve\__init__.py", line 381, in __generate_mar_file
    raise Exception("Unable to create mar file")
Exception: Unable to create mar file

Pls let me know how to proceed further..

[shrinath-suresh]

@saichandrapandraju Apologies for the delay. The above trace looks like a path issue. The torch-model-archiver is unable to find the relevant files.

The generic exception Exception: Unable to create mar file is not giving much information about the same. I have addressed it in the following PR https://github.com/mlflow/mlflow-torchserve/pull/81

It would be released soon in next version of the plugin. or you can build the plugin from source.

For installation refer - https://github.com/mlflow/mlflow-torchserve/blob/master/README.md