Generate Coursier credentials from a netrc file

[nlewo]

A simple parser parse the user netrc file and generate a list of credentials that can be used by Coursier.

Fixes #483

[dmivankov]

There is a workaround - configure both netrc and default coursier location, like ~/.config/coursier/credentials.properties But very nice to have single source of truth :+1:

Seems to be safe to add extra --credentials https://get-coursier.io/docs/other-credentials default locations would still be considered

[google-cla[bot]]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google. In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google. In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[googlebot]

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

[google-cla[bot]]

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

[mrkkrp]

@nlewo Could you please tell @googlebot that you consent, like it says in the comment above?

[nlewo]

@googlebot I consent.

[shs96c]

@jin any idea why the clabot hasn't cleared the cla:no label?

[jin]

There are two commits in this PR, one each by @nlewo and @mrkkrp. I forced a rescan and the CLA bot says that it needs the consent from @mrkkrp.

@mrkkrp, could you respond with "@googlebot I consent." as well?

[mrkkrp]

@googlebot I consent.

[google-cla[bot]]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[mrkkrp]

Please hold on with merging for now, there may be a minor problem with this patch. I'll look into this today.

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[google-cla[bot]]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[mrkkrp]

I have pushed a new commit. Now the coursier_fetch and pinned_coursier_fetch rules inspect the filesystem to get credentials. To ensure that events like adding a .netrc file after an initial run have effect, we need to set the local attribute of the repository rules to True. I'm not totally sure what kind of undesirable consequences this may have though.

[mrkkrp]

I don't think that the CI failures are related, but I may be wrong.

[mrkkrp]

Nevermind, I removed the commit that adds local = True to the repo rules definition. I did not fix the issue. Apparently bazel clean is necessary in both cases. IMHO, the patch is good to go.

[shs96c]

@mrkkrp This error looks valid: https://buildkite.com/bazel/rules-jvm-external/builds/1767#1e42648b-8170-4c69-ade7-99d88af96a7a/101-112

[dmivankov]

Downloading https://releases.bazel.build/1.0.1/release/bazel-1.0.1-linux-x86_64

could be that read_netrc isn't available in bazel 1.0.1

[FaBrand]

Is this still being pursued? I had to patch coursier as well to deal with authentication. One followup that i encountered after this is that the user that did the actual pinning would end up in the pinned.json file:

 "url": "https://<user>@<repository>/

I could remove it by reusing remove_auth_from_url for the url part in the parsed dep_tree.

+    for artifact in dep_tree["dependencies"]:
+        url = artifact.get("url", None)
+        if url == None:
+            continue
+        artifact.update({"url": remove_auth_from_url(url)})

As well as an additional method to do the same with the file path. That one contained the user as well

"file": "v1/https/<username>%40<repo_url>/

[shs96c]

This is still something we want to land.

[nlewo]

fyi, I no longer work on this topic so i could not terminate this work :(

[shs96c]

This has been superseded by #778