MLOpsPython
MLOpsPython copied to clipboard
Published
20 hours ago •
microsoft
microsoft
Ubuntu-latest (20.04.3) in Azure DevOps and latest azure-ml result in .NET Core 2.1 error
Hi,
I followed your templates to setup the CI/CD for Azure ML with Azure DevOps. However, I receive the following error. It seems to be related to the update of "Ubuntu-latest" from 20.04.3 to 20.04.2
Seems to be somehow related to https://github.com/Azure/MachineLearningNotebooks/issues/1003 However, I am just wondering what's the recommended fix since I followed your templates.
Here the logs:
2021-11-13T10:04:08.0321354Z ##[section]Starting: Publish Pipeline
2021-11-13T10:04:08.0329746Z ==============================================================================
2021-11-13T10:04:08.0330086Z Task : Azure CLI
2021-11-13T10:04:08.0330556Z Description : Run Azure CLI commands against an Azure subscription in a Shell script when running on Linux agent or Batch script when running on Windows agent.
2021-11-13T10:04:08.0331000Z Version : 1.164.0
2021-11-13T10:04:08.0331251Z Author : Microsoft Corporation
2021-11-13T10:04:08.0331625Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
2021-11-13T10:04:08.0332017Z ==============================================================================
2021-11-13T10:04:08.2124426Z [command]/usr/bin/az --version
2021-11-13T10:04:09.1258159Z azure-cli 2.30.0
2021-11-13T10:04:09.1265234Z
2021-11-13T10:04:09.1265701Z core 2.30.0
2021-11-13T10:04:09.1379430Z telemetry 1.0.6
2021-11-13T10:04:09.1379811Z
2021-11-13T10:04:09.1380084Z Extensions:
2021-11-13T10:04:09.1380272Z
2021-11-13T10:04:09.1381190Z azure-devops 0.21.0
2021-11-13T10:04:09.1381367Z
2021-11-13T10:04:09.1429224Z Python location '/opt/az/bin/python3'
2021-11-13T10:04:09.1429799Z Please let us know how we are doing: https://aka.ms/azureclihats
2021-11-13T10:04:09.1430591Z and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
2021-11-13T10:04:09.1431241Z Extensions directory '/opt/az/azcliextensions'
2021-11-13T10:04:09.1431440Z
2021-11-13T10:04:09.1431895Z Python (Linux) 3.6.10 (default, Oct 29 2021, 10:11:34)
2021-11-13T10:04:09.1432260Z [GCC 9.3.0]
2021-11-13T10:04:09.1432383Z
2021-11-13T10:04:09.1432706Z Legal docs and information: aka.ms/AzureCliLegal
2021-11-13T10:04:09.1433359Z
2021-11-13T10:04:09.1433509Z
2021-11-13T10:04:09.1434045Z Your CLI is up-to-date.
2021-11-13T10:04:09.1436383Z Setting AZURE_CONFIG_DIR env variable to: /home/vsts/work/_temp/.azclitask
2021-11-13T10:04:09.1438056Z Setting active cloud to: AzureCloud
2021-11-13T10:04:09.1446154Z [command]/usr/bin/az cloud set -n AzureCloud
2021-11-13T10:04:11.1164663Z [command]/usr/bin/az login --service-principal -u *** --password=*** --tenant ***
2021-11-13T10:04:13.2033470Z [
2021-11-13T10:04:13.2033789Z {
2021-11-13T10:04:13.2034084Z "cloudName": "AzureCloud",
2021-11-13T10:04:13.2035924Z "homeTenantId": "***",
2021-11-13T10:04:13.2036540Z "id": "484d48d1-7d2c-4d72-a1e1-e598af02f1da",
2021-11-13T10:04:13.2036880Z "isDefault": true,
2021-11-13T10:04:13.2037192Z "managedByTenants": [],
2021-11-13T10:04:13.2037501Z "name": "StorSimple",
2021-11-13T10:04:13.2037801Z "state": "Enabled",
2021-11-13T10:04:13.2038727Z "tenantId": "***",
2021-11-13T10:04:13.2039062Z "user": {
2021-11-13T10:04:13.2039784Z "name": "***",
2021-11-13T10:04:13.2040138Z "type": "servicePrincipal"
2021-11-13T10:04:13.2040419Z }
2021-11-13T10:04:13.2040697Z }
2021-11-13T10:04:13.2040935Z ]
2021-11-13T10:04:13.2057544Z [command]/usr/bin/az account set --subscription 484d48d1-7d2c-4d72-a1e1-e598af02f1da
2021-11-13T10:04:13.5171369Z [command]/usr/bin/bash /home/vsts/work/_temp/azureclitaskscript1636797848205.sh
2021-11-13T10:04:15.5707175Z Performing interactive authentication. Please follow the instructions on the terminal.
2021-11-13T10:04:15.5708163Z To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code ARCV3S5BK to authenticate.
2021-11-13T10:05:07.0509645Z No Python version provided, defaulting to "3.6.2"
2021-11-13T10:05:11.2373497Z Traceback (most recent call last):
2021-11-13T10:05:11.2375193Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/dotnetcore2/runtime.py", line 271, in attempt_get_deps
2021-11-13T10:05:11.2376412Z blob_deps_to_file()
2021-11-13T10:05:11.2379897Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/dotnetcore2/runtime.py", line 263, in blob_deps_to_file
2021-11-13T10:05:11.2381015Z blob = request.urlopen(deps_url, context=ssl_context)
2021-11-13T10:05:11.2381509Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/urllib/request.py", line 222, in urlopen
2021-11-13T10:05:11.2381977Z return opener.open(url, data, timeout)
2021-11-13T10:05:11.2382437Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/urllib/request.py", line 531, in open
2021-11-13T10:05:11.2382882Z response = meth(req, response)
2021-11-13T10:05:11.2383549Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/urllib/request.py", line 640, in http_response
2021-11-13T10:05:11.2384758Z response = self.parent.error(
2021-11-13T10:05:11.2385222Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/urllib/request.py", line 569, in error
2021-11-13T10:05:11.2385613Z return self._call_chain(*args)
2021-11-13T10:05:11.2385992Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/urllib/request.py", line 502, in _call_chain
2021-11-13T10:05:11.2386389Z result = func(*args)
2021-11-13T10:05:11.2386797Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/urllib/request.py", line 649, in http_error_default
2021-11-13T10:05:11.2387234Z raise HTTPError(req.full_url, code, msg, hdrs, fp)
2021-11-13T10:05:11.2387580Z urllib.error.HTTPError: HTTP Error 404: Not Found
2021-11-13T10:05:11.2387774Z
2021-11-13T10:05:11.2388098Z During handling of the above exception, another exception occurred:
2021-11-13T10:05:11.2388307Z
2021-11-13T10:05:11.2388688Z Traceback (most recent call last):
2021-11-13T10:05:11.2389139Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/runpy.py", line 194, in _run_module_as_main
2021-11-13T10:05:11.2389611Z return _run_code(code, main_globals, None,
2021-11-13T10:05:11.2390066Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/runpy.py", line 87, in _run_code
2021-11-13T10:05:11.2390489Z exec(code, run_globals)
2021-11-13T10:05:11.2390906Z File "/home/vsts/work/1/s/ml_service/pipelines/build_train_pipeline.py", line 163, in <module>
2021-11-13T10:05:11.2391342Z main(parser.parse_args())
2021-11-13T10:05:11.2392025Z File "/home/vsts/work/1/s/ml_service/pipelines/build_train_pipeline.py", line 77, in main
2021-11-13T10:05:11.2392397Z input_dataset = get_dataset(
2021-11-13T10:05:11.2392738Z File "/home/vsts/work/1/s/ml_service/util/dataset.py", line 41, in get_dataset
2021-11-13T10:05:11.2393125Z dataset = Dataset.Tabular.from_delimited_files(
2021-11-13T10:05:11.2393936Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/data/_loggerfactory.py", line 129, in wrapper
2021-11-13T10:05:11.2394382Z return func(*args, **kwargs)
2021-11-13T10:05:11.2395047Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/data/dataset_factory.py", line 352, in from_delimited_files
2021-11-13T10:05:11.2395517Z dataflow = dataprep().read_csv(path,
2021-11-13T10:05:11.2396186Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/readers.py", line 100, in read_csv
2021-11-13T10:05:11.2396926Z df = Dataflow._path_to_get_files_block(path, archive_options)
2021-11-13T10:05:11.2397696Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/dataflow.py", line 2426, in _path_to_get_files_block
2021-11-13T10:05:11.2398178Z return datastore_to_dataflow(path)
2021-11-13T10:05:11.2399059Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/_datastore_helper.py", line 27, in datastore_to_dataflow
2021-11-13T10:05:11.2399640Z datastore, datastore_value = get_datastore_value(source)
2021-11-13T10:05:11.2400468Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/_datastore_helper.py", line 80, in get_datastore_value
2021-11-13T10:05:11.2400971Z _set_auth_type(workspace)
2021-11-13T10:05:11.2402050Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/_datastore_helper.py", line 143, in _set_auth_type
2021-11-13T10:05:11.2402795Z get_engine_api().set_aml_auth(SetAmlAuthMessageArgument(AuthType.DERIVED, json.dumps(auth)))
2021-11-13T10:05:11.2403595Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/api.py", line 19, in get_engine_api
2021-11-13T10:05:11.2404026Z _engine_api = EngineAPI()
2021-11-13T10:05:11.2404693Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/api.py", line 68, in __init__
2021-11-13T10:05:11.2405156Z self._message_channel = launch_engine()
2021-11-13T10:05:11.2405848Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/engine.py", line 333, in launch_engine
2021-11-13T10:05:11.2406335Z dependencies_path = runtime.ensure_dependencies()
2021-11-13T10:05:11.2406993Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/dotnetcore2/runtime.py", line 285, in ensure_dependencies
2021-11-13T10:05:11.2407455Z if not attempt_get_deps():
2021-11-13T10:05:11.2408109Z File "/usr/share/miniconda/envs/scikit_env_ci/lib/python3.8/site-packages/dotnetcore2/runtime.py", line 279, in attempt_get_deps
2021-11-13T10:05:11.2409153Z raise NotImplementedError(err_msg + '\n' + _unsupported_help_msg)
2021-11-13T10:05:11.2409631Z NotImplementedError: Linux distribution ubuntu 20.04 does not have automatic support.
2021-11-13T10:05:11.2410158Z .NET Core 2.1 can still be used via dotnetcore2 if the required dependencies are installed.
2021-11-13T10:05:11.2410919Z Visit https://aka.ms/dotnet-install-linux for Linux distro specific .NET Core install instructions.
2021-11-13T10:05:11.2411914Z Follow your distro specific instructions to install `dotnet-runtime-*` and replace `*` with `2.1`.
2021-11-13T10:05:11.2412145Z
2021-11-13T10:05:11.2430036Z You have logged in. Now let us find all the subscriptions to which you have access...
2021-11-13T10:05:11.2430610Z Interactive authentication successfully completed.
2021-11-13T10:05:11.2430942Z get_workspace:
2021-11-13T10:05:11.2432138Z Workspace.create(name='xxaml-machinelearning', subscription_id='484d48d1-7d2c-4d72-a1e1-e598af02f1da', resource_group='XX_Machine_Learning')
2021-11-13T10:05:11.2433121Z Found existing compute target cpu-cluster-ds3 so using it.
2021-11-13T10:05:11.2433716Z aml_compute:
2021-11-13T10:05:11.2435323Z AmlCompute(workspace=Workspace.create(name=xxaml-machinelearning', subscription_id='XX', resource_group='XX_Machine_Learning'), name=cpu-cluster-ds3, id=/subscriptions/XX/resourceGroups/XX_Machine_Learning/providers/Microsoft.MachineLearningServices/workspaces/xxaml-machinelearning/computes/cpu-cluster-ds3, type=AmlCompute, provisioning_state=Succeeded, location=westus, tags=None)
2021-11-13T10:05:11.2436299Z Environment(Name: scikit_env,
2021-11-13T10:05:11.2436622Z Version: None)
2021-11-13T10:05:11.3988749Z ##[error]Script failed with error: Error: The process '/usr/bin/bash' failed with exit code 1
2021-11-13T10:05:11.4028907Z [command]/usr/bin/az account clear
2021-11-13T10:05:11.8318676Z ##[section]Finishing: Publish Pipeline
Here the condo environment:
2021-11-13T10:03:48.5432958Z # packages in environment at /usr/share/miniconda/envs/scikit_env_ci:
2021-11-13T10:03:48.5433894Z #
2021-11-13T10:03:48.5434462Z # Name Version Build Channel
2021-11-13T10:03:48.5435083Z _libgcc_mutex 0.1 main
2021-11-13T10:03:48.5436213Z _openmp_mutex 4.5 1_gnu
2021-11-13T10:03:48.5438094Z adal 1.2.7 pypi_0 pypi
2021-11-13T10:03:48.5438816Z applicationinsights 0.11.10 pypi_0 pypi
2021-11-13T10:03:48.5439447Z attrs 21.2.0 pypi_0 pypi
2021-11-13T10:03:48.5441001Z azure-common 1.1.27 pypi_0 pypi
2021-11-13T10:03:48.5441945Z azure-core 1.20.1 pypi_0 pypi
2021-11-13T10:03:48.5442857Z azure-graphrbac 0.61.1 pypi_0 pypi
2021-11-13T10:03:48.5443699Z azure-identity 1.4.1 pypi_0 pypi
2021-11-13T10:03:48.5445701Z azure-mgmt-authorization 0.61.0 pypi_0 pypi
2021-11-13T10:03:48.5446689Z azure-mgmt-containerregistry 8.2.0 pypi_0 pypi
2021-11-13T10:03:48.5447585Z azure-mgmt-core 1.3.0 pypi_0 pypi
2021-11-13T10:03:48.5448452Z azure-mgmt-keyvault 2.2.0 pypi_0 pypi
2021-11-13T10:03:48.5449321Z azure-mgmt-resource 13.0.0 pypi_0 pypi
2021-11-13T10:03:48.5450389Z azure-mgmt-storage 11.2.0 pypi_0 pypi
2021-11-13T10:03:48.5451256Z azureml-automl-core 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5452006Z azureml-core 1.26.0.post1 pypi_0 pypi
2021-11-13T10:03:48.5452750Z azureml-dataprep 2.13.2 pypi_0 pypi
2021-11-13T10:03:48.5453908Z azureml-dataprep-native 32.0.0 pypi_0 pypi
2021-11-13T10:03:48.5454724Z azureml-dataprep-rslex 1.11.2 pypi_0 pypi
2021-11-13T10:03:48.5455520Z azureml-dataset-runtime 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5456379Z azureml-pipeline 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5457424Z azureml-pipeline-core 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5458246Z azureml-pipeline-steps 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5459108Z azureml-sdk 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5460088Z azureml-telemetry 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5461136Z azureml-train 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5462084Z azureml-train-automl-client 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5462910Z azureml-train-core 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5463781Z azureml-train-restclients-hyperdrive 1.26.0 pypi_0 pypi
2021-11-13T10:03:48.5464701Z backports-tempfile 1.0 pypi_0 pypi
2021-11-13T10:03:48.5465526Z backports-weakref 1.0.post1 pypi_0 pypi
2021-11-13T10:03:48.5469243Z blas 1.0 mkl
2021-11-13T10:03:48.5469668Z brotli 1.0.9 he6710b0_2
2021-11-13T10:03:48.5470598Z ca-certificates 2021.10.26 h06a4308_2
2021-11-13T10:03:48.5470994Z certifi 2021.10.8 py38h06a4308_0
2021-11-13T10:03:48.5471362Z cffi 1.15.0 pypi_0 pypi
2021-11-13T10:03:48.5471857Z chardet 3.0.4 pypi_0 pypi
2021-11-13T10:03:48.5472256Z cloudpickle 1.6.0 pypi_0 pypi
2021-11-13T10:03:48.5472933Z contextlib2 21.6.0 pypi_0 pypi
2021-11-13T10:03:48.5473338Z coverage 6.1.2 pypi_0 pypi
2021-11-13T10:03:48.5473724Z cryptography 3.4.8 pypi_0 pypi
2021-11-13T10:03:48.5474114Z cycler 0.10.0 py38_0
2021-11-13T10:03:48.5474501Z dbus 1.13.18 hb2f20db_0
2021-11-13T10:03:48.5474895Z distro 1.6.0 pypi_0 pypi
2021-11-13T10:03:48.5475280Z docker 5.0.3 pypi_0 pypi
2021-11-13T10:03:48.5475916Z dotnetcore2 2.1.21 pypi_0 pypi
2021-11-13T10:03:48.5476461Z entrypoints 0.3 pypi_0 pypi
2021-11-13T10:03:48.5476881Z expat 2.4.1 h2531618_2
2021-11-13T10:03:48.5477285Z flake8 3.7.9 pypi_0 pypi
2021-11-13T10:03:48.5477984Z flake8-formatter-junit-xml 0.0.6 pypi_0 pypi
2021-11-13T10:03:48.5478425Z fontconfig 2.13.1 h6c09931_0
2021-11-13T10:03:48.5478839Z fonttools 4.25.0 pyhd3eb1b0_0
2021-11-13T10:03:48.5479236Z freetype 2.11.0 h70c0345_0
2021-11-13T10:03:48.5479657Z fusepy 3.0.1 pypi_0 pypi
2021-11-13T10:03:48.5480078Z giflib 5.2.1 h7b6447c_0
2021-11-13T10:03:48.5480488Z glib 2.69.1 h5202010_0
2021-11-13T10:03:48.5481090Z gst-plugins-base 1.14.0 h8213a91_2
2021-11-13T10:03:48.5481520Z gstreamer 1.14.0 h28cd5cc_2
2021-11-13T10:03:48.5481933Z icu 58.2 he6710b0_3
2021-11-13T10:03:48.5482565Z idna 2.10 pypi_0 pypi
2021-11-13T10:03:48.5483678Z inference-schema 1.1.0 pypi_0 pypi
2021-11-13T10:03:48.5484542Z intel-openmp 2021.4.0 h06a4308_3561
2021-11-13T10:03:48.5484974Z isodate 0.6.0 pypi_0 pypi
2021-11-13T10:03:48.5487294Z jeepney 0.7.1 pypi_0 pypi
2021-11-13T10:03:48.5487736Z jmespath 0.10.0 pypi_0 pypi
2021-11-13T10:03:48.5488167Z joblib 0.17.0 pypi_0 pypi
2021-11-13T10:03:48.5488590Z jpeg 9d h7f8727e_0
2021-11-13T10:03:48.5489025Z jsonpickle 2.0.0 pypi_0 pypi
2021-11-13T10:03:48.5489749Z junit-xml 1.9 pypi_0 pypi
2021-11-13T10:03:48.5490194Z kiwisolver 1.3.1 py38h2531618_0
2021-11-13T10:03:48.5490609Z lcms2 2.12 h3be6417_0
2021-11-13T10:03:48.5491336Z ld_impl_linux-64 2.35.1 h7274673_9
2021-11-13T10:03:48.5491744Z libffi 3.3 he6710b0_2
2021-11-13T10:03:48.5492429Z libgcc-ng 9.3.0 h5101ec6_17
2021-11-13T10:03:48.5493965Z libgfortran-ng 7.5.0 ha8ba4b0_17
2021-11-13T10:03:48.5494402Z libgfortran4 7.5.0 ha8ba4b0_17
2021-11-13T10:03:48.5494772Z libgomp 9.3.0 h5101ec6_17
2021-11-13T10:03:48.5495157Z libpng 1.6.37 hbc83047_0
2021-11-13T10:03:48.5495853Z libstdcxx-ng 9.3.0 hd4cf53a_17
2021-11-13T10:03:48.5496267Z libtiff 4.2.0 h85742a9_0
2021-11-13T10:03:48.5496770Z libuuid 1.0.3 h7f8727e_2
2021-11-13T10:03:48.5497183Z libwebp 1.2.0 h89dd481_0
2021-11-13T10:03:48.5497778Z libwebp-base 1.2.0 h27cfd23_0
2021-11-13T10:03:48.5498455Z libxcb 1.14 h7b6447c_0
2021-11-13T10:03:48.5498855Z libxml2 2.9.12 h03d6c58_0
2021-11-13T10:03:48.5499275Z llvmlite 0.37.0 pypi_0 pypi
2021-11-13T10:03:48.5499929Z lz4-c 1.9.3 h295c915_1
2021-11-13T10:03:48.5500360Z matplotlib 3.4.3 py38h06a4308_0
2021-11-13T10:03:48.5500939Z matplotlib-base 3.4.3 py38hbbc1b5f_0
2021-11-13T10:03:48.5501377Z mccabe 0.6.1 pypi_0 pypi
2021-11-13T10:03:48.5502083Z mkl 2021.4.0 h06a4308_640
2021-11-13T10:03:48.5502794Z mkl-service 2.4.0 py38h7f8727e_0
2021-11-13T10:03:48.5503224Z mkl_fft 1.3.1 py38hd3c417c_0
2021-11-13T10:03:48.5503634Z mkl_random 1.2.2 py38h51133e4_0
2021-11-13T10:03:48.5504237Z more-itertools 8.11.0 pypi_0 pypi
2021-11-13T10:03:48.5504675Z msal 1.16.0 pypi_0 pypi
2021-11-13T10:03:48.5505295Z msal-extensions 0.2.2 pypi_0 pypi
2021-11-13T10:03:48.5505845Z msrest 0.6.21 pypi_0 pypi
2021-11-13T10:03:48.5506262Z msrestazure 0.6.4 pypi_0 pypi
2021-11-13T10:03:48.5506787Z munkres 1.1.4 py_0
2021-11-13T10:03:48.5507166Z ncurses 6.3 h7f8727e_2
2021-11-13T10:03:48.5507766Z ndg-httpsclient 0.5.1 pypi_0 pypi
2021-11-13T10:03:48.5508213Z numba 0.54.1 pypi_0 pypi
2021-11-13T10:03:48.5508644Z numpy 1.20.3 pypi_0 pypi
2021-11-13T10:03:48.5509057Z oauthlib 3.1.1 pypi_0 pypi
2021-11-13T10:03:48.5509475Z olefile 0.46 pyhd3eb1b0_0
2021-11-13T10:03:48.5509894Z openssl 1.1.1l h7f8727e_0
2021-11-13T10:03:48.5510299Z packaging 21.2 pypi_0 pypi
2021-11-13T10:03:48.5510720Z pandas 1.1.5 py38ha9443f7_0
2021-11-13T10:03:48.5511138Z pathspec 0.9.0 pypi_0 pypi
2021-11-13T10:03:48.5511559Z pcre 8.45 h295c915_0
2021-11-13T10:03:48.5511952Z pillow 8.4.0 py38h5aabda8_0
2021-11-13T10:03:48.5512373Z pip 20.3.3 py38h06a4308_0
2021-11-13T10:03:48.5512791Z pluggy 0.13.1 pypi_0 pypi
2021-11-13T10:03:48.5513219Z portalocker 1.7.1 pypi_0 pypi
2021-11-13T10:03:48.5513632Z py 1.11.0 pypi_0 pypi
2021-11-13T10:03:48.5514181Z pyarrow 1.0.1 pypi_0 pypi
2021-11-13T10:03:48.5514713Z pyasn1 0.4.8 pypi_0 pypi
2021-11-13T10:03:48.5515137Z pycodestyle 2.5.0 pypi_0 pypi
2021-11-13T10:03:48.5515565Z pycparser 2.21 pypi_0 pypi
2021-11-13T10:03:48.5515975Z pyflakes 2.1.1 pypi_0 pypi
2021-11-13T10:03:48.5516400Z pyjwt 2.3.0 pypi_0 pypi
2021-11-13T10:03:48.5516823Z pyopenssl 20.0.1 pypi_0 pypi
2021-11-13T10:03:48.5517255Z pyparsing 2.4.7 pypi_0 pypi
2021-11-13T10:03:48.5517657Z pyqt 5.9.2 py38h05f1152_4
2021-11-13T10:03:48.5518076Z pytest 5.4.3 pypi_0 pypi
2021-11-13T10:03:48.5518708Z pytest-cov 2.8.1 pypi_0 pypi
2021-11-13T10:03:48.5519144Z python 3.8.12 h12debd9_0
2021-11-13T10:03:48.5519898Z python-dateutil 2.8.2 pyhd3eb1b0_0
2021-11-13T10:03:48.5520525Z python-dotenv 0.12.0 pypi_0 pypi
2021-11-13T10:03:48.5520960Z pytz 2021.3 pyhd3eb1b0_0
2021-11-13T10:03:48.5521379Z pyyaml 5.3 pypi_0 pypi
2021-11-13T10:03:48.5521782Z qt 5.9.7 h5867ecd_1
2021-11-13T10:03:48.5522191Z readline 8.1 h27cfd23_0
2021-11-13T10:03:48.5522749Z requests 2.23.0 pypi_0 pypi
2021-11-13T10:03:48.5523400Z requests-oauthlib 1.3.0 pypi_0 pypi
2021-11-13T10:03:48.5524008Z ruamel-yaml 0.17.4 pypi_0 pypi
2021-11-13T10:03:48.5524625Z ruamel-yaml-clib 0.2.6 pypi_0 pypi
2021-11-13T10:03:48.5525353Z scikit-learn 0.23.2 py38h0573a6f_0
2021-11-13T10:03:48.5525773Z scipy 1.7.1 py38h292c36d_2
2021-11-13T10:03:48.5526166Z secretstorage 3.3.1 pypi_0 pypi
2021-11-13T10:03:48.5526697Z setuptools 58.0.4 py38h06a4308_0
2021-11-13T10:03:48.5527117Z shap 0.37.0 pypi_0 pypi
2021-11-13T10:03:48.5527533Z sip 4.19.13 py38he6710b0_0
2021-11-13T10:03:48.5527929Z six 1.16.0 pyhd3eb1b0_0
2021-11-13T10:03:48.5528355Z slicer 0.0.3 pypi_0 pypi
2021-11-13T10:03:48.5528773Z sqlite 3.36.0 hc218d9a_0
2021-11-13T10:03:48.5529180Z threadpoolctl 2.2.0 pyh0d69192_0
2021-11-13T10:03:48.5529701Z tk 8.6.11 h1ccaba5_0
2021-11-13T10:03:48.5530098Z tornado 6.1 py38h27cfd23_0
2021-11-13T10:03:48.5530509Z tqdm 4.62.3 pypi_0 pypi
2021-11-13T10:03:48.5530922Z urllib3 1.25.11 pypi_0 pypi
2021-11-13T10:03:48.5531321Z wcwidth 0.2.5 pypi_0 pypi
2021-11-13T10:03:48.5531923Z websocket-client 1.2.1 pypi_0 pypi
2021-11-13T10:03:48.5532346Z wheel 0.37.0 pyhd3eb1b0_1
2021-11-13T10:03:48.5532997Z wrapt 1.11.1 pypi_0 pypi
2021-11-13T10:03:48.5533401Z xz 5.2.5 h7b6447c_0
2021-11-13T10:03:48.5533785Z zlib 1.2.11 h7b6447c_3
2021-11-13T10:03:48.5534286Z zstd 1.4.9 haebb681_0
2021-11-13T10:03:48.9625266Z Package Version
2021-11-13T10:03:48.9626464Z ------------------------------------ ------------
2021-11-13T10:03:48.9626879Z adal 1.2.7
2021-11-13T10:03:48.9627228Z applicationinsights 0.11.10
2021-11-13T10:03:48.9627556Z attrs 21.2.0
2021-11-13T10:03:48.9628093Z azure-common 1.1.27
2021-11-13T10:03:48.9628615Z azure-core 1.20.1
2021-11-13T10:03:48.9629220Z azure-graphrbac 0.61.1
2021-11-13T10:03:48.9629676Z azure-identity 1.4.1
2021-11-13T10:03:48.9630149Z azure-mgmt-authorization 0.61.0
2021-11-13T10:03:48.9630634Z azure-mgmt-containerregistry 8.2.0
2021-11-13T10:03:48.9631100Z azure-mgmt-core 1.3.0
2021-11-13T10:03:48.9631550Z azure-mgmt-keyvault 2.2.0
2021-11-13T10:03:48.9632154Z azure-mgmt-resource 13.0.0
2021-11-13T10:03:48.9632782Z azure-mgmt-storage 11.2.0
2021-11-13T10:03:48.9633432Z azureml-automl-core 1.26.0
2021-11-13T10:03:48.9634384Z azureml-core 1.26.0.post1
2021-11-13T10:03:48.9634951Z azureml-dataprep 2.13.2
2021-11-13T10:03:48.9635490Z azureml-dataprep-native 32.0.0
2021-11-13T10:03:48.9636378Z azureml-dataprep-rslex 1.11.2
2021-11-13T10:03:48.9636835Z azureml-dataset-runtime 1.26.0
2021-11-13T10:03:48.9637306Z azureml-pipeline 1.26.0
2021-11-13T10:03:48.9637778Z azureml-pipeline-core 1.26.0
2021-11-13T10:03:48.9638232Z azureml-pipeline-steps 1.26.0
2021-11-13T10:03:48.9638923Z azureml-sdk 1.26.0
2021-11-13T10:03:48.9639400Z azureml-telemetry 1.26.0
2021-11-13T10:03:48.9639994Z azureml-train 1.26.0
2021-11-13T10:03:48.9640478Z azureml-train-automl-client 1.26.0
2021-11-13T10:03:48.9640978Z azureml-train-core 1.26.0
2021-11-13T10:03:48.9641481Z azureml-train-restclients-hyperdrive 1.26.0
2021-11-13T10:03:48.9641842Z backports.tempfile 1.0
2021-11-13T10:03:48.9642172Z backports.weakref 1.0.post1
2021-11-13T10:03:48.9642645Z certifi 2021.10.8
2021-11-13T10:03:48.9643126Z cffi 1.15.0
2021-11-13T10:03:48.9643486Z chardet 3.0.4
2021-11-13T10:03:48.9643829Z cloudpickle 1.6.0
2021-11-13T10:03:48.9644187Z contextlib2 21.6.0
2021-11-13T10:03:48.9644545Z coverage 6.1.2
2021-11-13T10:03:48.9644894Z cryptography 3.4.8
2021-11-13T10:03:48.9645255Z cycler 0.10.0
2021-11-13T10:03:48.9645614Z distro 1.6.0
2021-11-13T10:03:48.9646087Z docker 5.0.3
2021-11-13T10:03:48.9646533Z dotnetcore2 2.1.21
2021-11-13T10:03:48.9646870Z entrypoints 0.3
2021-11-13T10:03:48.9647207Z flake8 3.7.9
2021-11-13T10:03:48.9647716Z flake8-formatter-junit-xml 0.0.6
2021-11-13T10:03:48.9648054Z fonttools 4.25.0
2021-11-13T10:03:48.9648392Z fusepy 3.0.1
2021-11-13T10:03:48.9648729Z idna 2.10
2021-11-13T10:03:48.9649218Z inference-schema 1.1.0
2021-11-13T10:03:48.9649551Z isodate 0.6.0
2021-11-13T10:03:48.9649888Z jeepney 0.7.1
2021-11-13T10:03:48.9650226Z jmespath 0.10.0
2021-11-13T10:03:48.9650571Z joblib 0.17.0
2021-11-13T10:03:48.9650891Z jsonpickle 2.0.0
2021-11-13T10:03:48.9651378Z junit-xml 1.9
2021-11-13T10:03:48.9651727Z kiwisolver 1.3.1
2021-11-13T10:03:48.9652050Z llvmlite 0.37.0
2021-11-13T10:03:48.9652386Z matplotlib 3.4.3
2021-11-13T10:03:48.9653264Z mccabe 0.6.1
2021-11-13T10:03:48.9653874Z mkl-fft 1.3.1
2021-11-13T10:03:48.9654397Z mkl-random 1.2.2
2021-11-13T10:03:48.9655041Z mkl-service 2.4.0
2021-11-13T10:03:48.9655560Z more-itertools 8.11.0
2021-11-13T10:03:48.9656046Z msal 1.16.0
2021-11-13T10:03:48.9656658Z msal-extensions 0.2.2
2021-11-13T10:03:48.9656994Z msrest 0.6.21
2021-11-13T10:03:48.9657322Z msrestazure 0.6.4
2021-11-13T10:03:48.9657638Z munkres 1.1.4
2021-11-13T10:03:48.9658086Z ndg-httpsclient 0.5.1
2021-11-13T10:03:48.9658417Z numba 0.54.1
2021-11-13T10:03:48.9658736Z numpy 1.20.3
2021-11-13T10:03:48.9659035Z oauthlib 3.1.1
2021-11-13T10:03:48.9659525Z olefile 0.46
2021-11-13T10:03:48.9659843Z packaging 21.2
2021-11-13T10:03:48.9660158Z pandas 1.1.5
2021-11-13T10:03:48.9660457Z pathspec 0.9.0
2021-11-13T10:03:48.9660773Z Pillow 8.4.0
2021-11-13T10:03:48.9661089Z pip 20.3.3
2021-11-13T10:03:48.9661408Z pluggy 0.13.1
2021-11-13T10:03:48.9661707Z portalocker 1.7.1
2021-11-13T10:03:48.9662265Z py 1.11.0
2021-11-13T10:03:48.9662722Z pyarrow 1.0.1
2021-11-13T10:03:48.9663193Z pyasn1 0.4.8
2021-11-13T10:03:48.9663535Z pycodestyle 2.5.0
2021-11-13T10:03:48.9663893Z pycparser 2.21
2021-11-13T10:03:48.9664251Z pyflakes 2.1.1
2021-11-13T10:03:48.9664613Z PyJWT 2.3.0
2021-11-13T10:03:48.9665078Z pyOpenSSL 20.0.1
2021-11-13T10:03:48.9665421Z pyparsing 2.4.7
2021-11-13T10:03:48.9665764Z pytest 5.4.3
2021-11-13T10:03:48.9666511Z pytest-cov 2.8.1
2021-11-13T10:03:48.9666988Z python-dateutil 2.8.2
2021-11-13T10:03:48.9667457Z python-dotenv 0.12.0
2021-11-13T10:03:48.9667786Z pytz 2021.3
2021-11-13T10:03:48.9668094Z PyYAML 5.3
2021-11-13T10:03:48.9668410Z requests 2.23.0
2021-11-13T10:03:48.9668871Z requests-oauthlib 1.3.0
2021-11-13T10:03:48.9669199Z ruamel.yaml 0.17.4
2021-11-13T10:03:48.9669500Z ruamel.yaml.clib 0.2.6
2021-11-13T10:03:48.9669960Z scikit-learn 0.23.2
2021-11-13T10:03:48.9670295Z scipy 1.7.1
2021-11-13T10:03:48.9670609Z SecretStorage 3.3.1
2021-11-13T10:03:48.9670910Z setuptools 58.0.4
2021-11-13T10:03:48.9671226Z shap 0.37.0
2021-11-13T10:03:48.9671545Z sip 4.19.13
2021-11-13T10:03:48.9671984Z six 1.16.0
2021-11-13T10:03:48.9672306Z slicer 0.0.3
2021-11-13T10:03:48.9672761Z threadpoolctl 2.2.0
2021-11-13T10:03:48.9673234Z tornado 6.1
2021-11-13T10:03:48.9673575Z tqdm 4.62.3
2021-11-13T10:03:48.9673938Z urllib3 1.25.11
2021-11-13T10:03:48.9674299Z wcwidth 0.2.5
2021-11-13T10:03:48.9674834Z websocket-client 1.2.1
2021-11-13T10:03:48.9675197Z wheel 0.37.0
2021-11-13T10:03:48.9675681Z wrapt 1.11.1
2021-11-13T10:03:49.1234031Z ##[section]Finishing: Create Anaconda environment
What's the pipeline, could you link to it?
PS: the shared link does seem to possibly be the same issue. Have you tried the solution there w/ dotnet core 2.1?
Hmm unfortunately I am Not able to share the pipeline :/
I also tried to install it via
- script: | source activate scikit_env_ci from dotnetcore2 import runtime bin_folder = runtime._get_bin_folder() missing_packages = runtime._gather_dependencies(bin_folder) print(missing_packages) displayName: Debug DotNet 2.1
or what is the recommended way?
I can try to reproduce it, but need some more info. Could you just give me more info on where you are on the process? I assume you've done the bootstrapping, is that correct? what's the name of the yml file where the error is happening?
It fails in this code: https://github.com/microsoft/MLOpsPython/blob/master/ml_service/pipelines/diabetes_regression_build_train_pipeline.py
As far as I understand this should be related to the azure ml dataset API. So probably https://github.com/microsoft/MLOpsPython/blob/master/ml_service/pipelines/diabetes_regression_build_train_pipeline.py#L77
The yaml is https://github.com/microsoft/MLOpsPython/blob/master/.pipelines/diabetes_regression-ci.yml
Perhaps it suffices to run the CI pipeline of this repo (if the issue has not been resolved by now): https://aidemos.visualstudio.com/MLOps/_build/results?buildId=5721&view=results