Disable .netrc conf for api client

[kartikgupta-db]

• Python requests library automatically picks up .netrc configurations for authentication. (https://github.com/psf/requests/issues/2773)
• This PR disables this behaviour to make it consistent with the docs (https://docs.databricks.com/dev-tools/cli/index.html#set-up-authentication)
• The fix uses a workaround which should be stable until the API for authentication in requests library changes.

[codecov-commenter]

Codecov Report

Merging #526 (0dc8638) into main (44ccc7d) will increase coverage by 3.86%. The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #526      +/-   ##
==========================================
+ Coverage   60.93%   64.79%   +3.86%     
==========================================
  Files          55       55              
  Lines        4669     4727      +58     
==========================================
+ Hits         2845     3063     +218     
+ Misses       1824     1664     -160     

Impacted Files	Coverage Δ
databricks_cli/utils.py	89.58% <0.00%> (-0.53%)	:arrow_down:
databricks_cli/unity_catalog/perms_cli.py	0.00% <0.00%> (ø)
databricks_cli/unity_catalog/metastore_cli.py	0.00% <0.00%> (ø)
databricks_cli/unity_catalog/delta_sharing_cli.py	0.00% <0.00%> (ø)
databricks_cli/configure/provider.py	95.97% <0.00%> (+0.09%)	:arrow_up:
databricks_cli/unity_catalog/uc_service.py	27.73% <0.00%> (+27.73%)	:arrow_up:
databricks_cli/unity_catalog/utils.py	30.76% <0.00%> (+30.76%)	:arrow_up:
databricks_cli/unity_catalog/api.py	47.97% <0.00%> (+47.97%)	:arrow_up:
databricks_cli/unity_catalog/lineage_cli.py	80.00% <0.00%> (+80.00%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[pietern]

We can't merge this as is. Reading through the linked issues, there's an opportunity to do this in a non-breaking way. The default auth handler checks the host and overrides a token from .netrc if present, regardless of existing authentication being configured. What we can do instead is fallback to reading from .netrc if nothing is configured. Then the case where a user has a .netrc and only specifies a hostname, the CLI will continue to work.

[pietern]

This is a nice workaround. In the body we could check for the Authorization header and only defer to .netrc if it isn't set.

https://github.com/psf/requests/issues/2773#issuecomment-174312831

[kartikgupta-db]

We can't merge this as is. Reading through the linked issues, there's an opportunity to do this in a non-breaking way. The default auth handler checks the host and overrides a token from .netrc if present, regardless of existing authentication being configured. What we can do instead is fallback to reading from .netrc if nothing is configured. Then the case where a user has a .netrc and only specifies a hostname, the CLI will continue to work.

The cli checks that there should be atleast 1 form of accepted configuration methods (spark config, environment variables or databrickscfg) are present. So there should be no cli user directly dependent on a .netrc.

For users using the api client directly from the sdk, the fallback should provide them the similar experience as they have now, with the caveat that now other forms of configurations will take precedence (earlier .netrc had the highest precedence). But this is an undocumented use case.

[pietern]

For users using the api client directly from the sdk, the fallback should provide them the similar experience as they have now, with the caveat that now other forms of configurations will take precedence (earlier .netrc had the highest precedence). But this is an undocumented use case.

It's one we need to call out in the changelog / release notes though ;-)