MlFlow load model form azure blob error: blob doesn't exist

[rjtshrm]

Willingness to contribute

Yes. I would be willing to contribute a fix for this bug with guidance from the MLflow community.

MLflow version

1.26.1

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 20.04):
• Python version: 3.10

Describe the problem

I am using mlflow with azure blob and psql. While training I can successfully logs, the PyTorch model on the Azure blob storage. However when I am trying to access the model for the inference I get error

mlflow.exceptions.MlflowException: The following failures occurred while downloading one or more artifacts from wasbs://[email protected]/111/353d98eb546d47ad851788d127e10a7e/artifacts: {'model': "ResourceNotFoundError('The specified blob does not exist.\nRequestId:f54d469e-801e-0038-102c-8f4ee4000000\nTime:2022-07-03T22:30:50.4136595Z\nErrorCode:BlobNotFound')"}

However, I can see the blob on the azure blob storage. What I found is that It is giving an error this for the folder, even when I try to access using azure storage python SDK. But if I set the final path with files such as conda.yaml or whatever, I can download the file but not the root folder.

Tracking information

No response

Code to reproduce issue

mlflow.set_tracking_uri("http://20.28.195.42/mlflow/")
mlflow.set_registry_uri("http://20.28.195.42/mlflow/")
self.model_path = f"runs:/{os.environ['RUN_ID']}/model"
self.model = mlflow.pytorch.load_model(self.model_path)

Other info / logs

No response

What component(s) does this bug affect?

• [X] area/artifacts: Artifact stores and artifact logging
• [ ] area/build: Build and test infrastructure for MLflow
• [ ] area/docs: MLflow documentation pages
• [ ] area/examples: Example code
• [ ] area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• [ ] area/models: MLmodel format, model serialization/deserialization, flavors
• [ ] area/pipelines: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates
• [ ] area/projects: MLproject format, project running backends
• [ ] area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• [ ] area/server-infra: MLflow Tracking server backend
• [X] area/tracking: Tracking Service, tracking client APIs, autologging

What interface(s) does this bug affect?

• [ ] area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• [ ] area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
• [ ] area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• [ ] area/windows: Windows support

What language(s) does this bug affect?

• [ ] language/r: R APIs and clients
• [ ] language/java: Java APIs and clients
• [ ] language/new: Proposals for new client languages

What integration(s) does this bug affect?

• [X] integrations/azure: Azure and Azure ML integrations
• [ ] integrations/sagemaker: SageMaker integrations
• [ ] integrations/databricks: Databricks integrations

[WeichenXu123]

@rjtshrm Thanks for reporting it ! Do you know the root cause of the issue ?

[sandeeps311]

same issue for me while downloading serving model from azure blob storage

[mlflow-automation]

@BenWilson2 @dbczumar @harupy @WeichenXu123 Please assign a maintainer and start triaging this issue.