MachineLearningNotebooks
MachineLearningNotebooks copied to clipboard
Azure Packages Break RunConfiguration().python.conda_dependencies
If you try to use azure-kusto-data or azure-kusto-ingest in your RunConfiguration() conda dependcies, either as PIP or conda installs, image build creation will fail and hang at the pip installation stage.
Consider this example:
# create a new runconfig object
aml_run_config = RunConfiguration()
# enable Docker
azureml.core.runconfig.DockerConfiguration(use_docker=True)
# use the aml_compute created above.
aml_run_config.target = compute_name
# set docker base image to the default CPU-based image
aml_run_config.environment.docker.base_image = None
aml_run_config.environment.docker.base_dockerfile = dockerfile
# Use conda_dependencies.yml to create a conda environment in the Docker image for execution
aml_run_config.environment.python.user_managed_dependencies = False
# Specify CondaDependencies obj, add necessary packages
aml_run_config.environment.python.conda_dependencies = CondaDependencies.create(
conda_packages=['pip', 'numpy', 'python==3.8', 'tenacity', 'pathlib', 'loguru'],
pip_packages=['azureml-core', 'azureml-sdk','azureml-dataset-runtime[pandas,fuse]',
'azureml-defaults', 'aiohttp', 'pyodbc', 'nest_asyncio', 'azure-identity', 'azure-keyvault-secrets', 'python-dotenv',
'azure-storage-file-share', 'azure-storage-file-datalake', 'pandas==1.4.1', 'lxml', 'azure-kusto-data', 'azure-kusto-ingest'])