secret scope names containing slashes cause failure

[neilbest-db]

databricks secrets list-scopes --profile bethesda2

returns:

Scope                                          Backend     KeyVault URL
---------------------------------------------  ----------  --------------
adaptive-planning-scope                        DATABRICKS  N/A
BI/Databricks/ForumUsers                       DATABRICKS  N/A
. . .

Secrets migration:

python ./migrate/export_db.py --secrets \                                                                                       (zmi-bi-databricks)
             --profile bethesda2 \
             --cluster-name "E2 migration" \
             --set-export-dir workflow/results/migrate/bethesda2/secrets

dies like this:

Get: https://bethesda2.cloud.databricks.com/api/2.0/secrets/list
Traceback (most recent call last):
  File "./migrate/export_db.py", line 297, in <module>
    main()
  File "./migrate/export_db.py", line 183, in main
    sc.log_all_secrets(args.cluster_name)
  File "/Users/neil.best/Documents/GitHub/BethesdaNet/zmi-bi-databricks/migrate/dbclient/SecretsClient.py", line 46, in log_all_secrets
    with open(scopes_logfile, 'w') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'workflow/results/migrate/bethesda2/secrets/secret_scopes/BI/Databricks/ForumUsers'

Export directory contains only:

find workflow/results/migrate/bethesda2/secrets -type f

workflow/results/migrate/bethesda2/secrets/secret_scopes/adaptive-planning-scope

The first scope succeeded but it looks like those slashes in the scope name need to be escaped or translated somehow for it to proceed.

[neilbest-db]

Looks like the fix goes here. Looking for the right idiom . . . my Python is not so good-looking (hat tip, Celia Cruz) . . .

[neilbest-db]

Would this work?

If the directory name contains a slash the OS module sees this as a :. Therefore you can create dirs containing slashes by specifying the dirname as followed os.mkdir(path + "dir:name"). The folder will be named dir/name.

[mrchristine]

Thanks for reporting this. Given that escaping names and other characters could cause a similar issue, I think a new design to how I store the data locally would be best to fix this. I'll look at fixing this w/ a redesign.