databricks-cli icon indicating copy to clipboard operation
databricks-cli copied to clipboard

issue with file system commands

Open gwadson opened this issue 2 years ago • 2 comments

Hi,

  • I'm using the community edition.
  • I installed the CLI version 0.16.6
  • may be related to #257

databricks fs ls is not returning all directories (FileStore, users, etc...) available on the workspace. The fs command returns only one directory, and when I try to list the directory, I get an error; see the image below

image

I created a directory using the CLI and it looks fine, but the directory is not visible in the workspace

image

As you can see, all the directories below and mydirectory are missing in the output of fs ls dbfs:/ image

do you have any idea why that's not working? Thanks

gwadson avatar Apr 13 '22 05:04 gwadson

I have a similar issue on version 0.17.1.

The "dbutils.fs.ls" command and the "databricks fs ls" cli command give me different results:

image

image

I'm unable to download files from dbfs to my locahost because the cli cannot "see" them.

jrosend avatar Aug 13 '22 04:08 jrosend

Hi! Are you sure that the CLI is configured such that it points to the same Databricks workspace where you're running the dbutils.fs command? For example, if you create a notebook in the UI and then run databricks workspace ls do you see it?

If not, please refer to the instructions at https://docs.databricks.com/dev-tools/cli/index.html#set-up-authentication Otherwise let me know and I can try to help further.

adrian-ionescu avatar Aug 15 '22 08:08 adrian-ionescu