dbldatagen icon indicating copy to clipboard operation
dbldatagen copied to clipboard
verified publisher icon databrickslabs

Uninstalling and reinstalling wheel on cluster running DBR 8.3 may fail

Open ronanstokes-db opened this issue 4 years ago • 1 comments

If you have a named cluster specification in your Databricks environment and it had the current or a previous build of the datagenerator installed, when you uninstall the library and reinstall it , it may fail

Expected Behavior

Uninstall followed by reinstall should succeed

Current Behavior

Uninstall followed by re-install may fail.

Workaround

  • make sure the wheel does not have a name like dbldatagen-0.2.0rc1-py3-none-any.whl (1) which may result from multiple downloads on the same machine
  • dont use a saved cluster definition - use a new cluster definition

Our plan is to move to a PIP based install which should make installation easier

Your Environment

  • dbldatagen version used: release candidate 2
  • Databricks Runtime version: Databricks 8.3
  • Cloud environment used: Azure

ronanstokes-db avatar Nov 12 '21 23:11 ronanstokes-db

You can also install in a Databricks notebook environment using the %pip command in a notebook cell. The %pip command does not illustrate the issue above.

To install the library via %pip, add a cell with the following content at the start of the notebook and execute it:

%pip install git+https://github.com/databrickslabs/dbldatagen

ronanstokes-db avatar Jan 14 '22 23:01 ronanstokes-db

Newest versions will be based on Databricks runtime 9.1 LTS or later

ronanstokes-db avatar Oct 04 '22 00:10 ronanstokes-db