versatile-data-kit icon indicating copy to clipboard operation
versatile-data-kit copied to clipboard

Google Collab Notebook job_input initialization fails on first try

Open antoniivanov opened this issue 2 years ago • 1 comments

Describe the bug

The first time job_input = VDK.get_initialized_job_input() is run in Google COllab notebook it raises an exception .

It succeeds if it's executed against (for a second time)

Steps To Reproduce Execute https://colab.research.google.com/github/antoniivanov/vdk-demo/blob/main/IPythonVDKInitBug.ipynb

Expected behavior

Succeeds on first try

antoniivanov avatar Oct 31 '23 08:10 antoniivanov

Looks to me like there is an issue with urllib3 when installing quickstart-vdk. This is during !pip install quickstart-vdk vdk-notebook vdk-ipython

It starts fine and sees there is a version higher than the requirement Requirement already satisfied: urllib3>=1.26.0 in /usr/local/lib/python3.10/dist-packages (from docker->vdk-server->quickstart-vdk) (2.0.7)

Then it downgrades it during the installation of quickstart-vdk Installing collected packages: click-spinner, aenum, urllib3, simplejson, jedi, ecs-logging, click_log, vdk-core, vdk-control-service-api, vdk-sqlite, vdk-notebook, vdk-logging-format, vdk-ipython, vdk-ingest-http, vdk-ingest-file, docker, vdk-control-api-auth, kubernetes, vdk-server, vdk-control-cli, vdk-plugin-control-cli, quickstart-vdk Attempting uninstall: urllib3 Found existing installation: urllib3 2.0.7 Uninstalling urllib3-2.0.7: Successfully uninstalled urllib3-2.0.7

Then while running job_input = VDK.get_initialized_job_input() It can't find the original urllib3 version ERROR:vdk.internal.plugin.plugin:[Errno 2] No such file or directory: '/usr/local/lib/python3.10/dist-packages/urllib3-2.0.7.dist-info/METADATA'

!pip list confirms that the current version is 1.26.18.

I will look into this further and see what I can do. Help is appreciated though

chrfoyer avatar Jan 04 '24 23:01 chrfoyer