What is the intended workflow to pin dependencies in a repository requiring authentication?

[hxtk]

In short, the issue is that configuring a repository with basic auth credentials and then pinning the dependency tree results in those basic auth credentials being embedded in maven_install.json, which is not a file intended to be edited manually. netrc authentication cannot be used until after the dependencies have already been pinned, so the intended sequence of operations to end up with a maven_install.json file which does not embed credentials and uses netrc for its authentication information is not clear to me.

The actual use case that I am trying to support is storing credentials in a way that allows me to commit all of my BAZEL files to version control without leaking those credentials or forcing all developers and service accounts to use the same credentials.

Below, I have detailed the steps that I personally took to work around this in case it helps to spot the point where I went wrong.

---

I have a Maven repository which requires authentication. This is a supported use-case noted in the usage section of the README, and I was able to download the dependencies.

However, this had two issues: first of all, I did not want to hard-code credentials (especially not in a file that would be committed to version control) and second of all I wanted to pin my dependencies.

When I pinned the dependencies, I expected to be able to use the netrc_lines or a netrc file to store credentials by either substituting them from environment variables or reading them from a .gitignore'd netrc file that each developer could have on their own machine.

This didn't work, though, because pinning the dependencies embeds the exact URLs as built from the repository definitions, including the credentials. I used search-and-replace to remove any instances of the credentials from the generated maven_install.json file, but this caused the file to fail checksum. I removed the checksum and the result was something that worked, but throws warnings every time.

I don't think I'm using this feature as intended, but it is not clear to me how I am meant to use the netrc parameters to add credentials if I cannot use netrc for credentials until I have pinned the dependencies and the dependencies can only be pinned with their credentials already embedded.

Although it doesn't strictly match the issue title, I would also be happy to be informed of any other intended mechanism for allowing credentials to be set per-developer rather than embedding them in version-controlled files.

[cheister]

If you don't put the credentials in the url and use a properties file for coursier can you pin the artifacts without having the username and password in the maven_install.json file?

[cheister]

Alternatively, the rules will add the entries from ~/.netrc file if it exists and you're not on Windows. If you add entries to your ~/.netrc file and remove the username:password from the mirror urls do you get the behavior you want?

[hxtk]

If you don't put the credentials in the url and use a properties file for coursier can you pin the artifacts without having the username and password in the maven_install.json file?

I was able to try this today. Using the coursier properties file does indeed allow me to interact with authenticated repos and pin my dependencies without embedding credentials.

Alternatively, the rules will add the entries from ~/.netrc file if it exists and you're not on Windows. If you add entries to your ~/.netrc file and remove the username:password from the mirror urls do you get the behavior you want?

This is what I ended up instructing devs to do in the README for installing the pinned dependencies, but it did not seem to help when initially pinning dependencies.

[phillcunnington]

I have the same issue and I find even though I have the properties file for coursier when pinning the dependencies the username is still present in the maven_install.json file in the url for the dependency.

[hxtk]

Point of clarification: my experience matches that of @phillcunnington

The passwords are not recorded, which makes it usable at the very least without leaking anything that is sensitive in my use case, but the usernames still exist which is certainly not ideal.

I've confirmed that even with the usernames embedded (and passwords absent), it is possible to authenticate as a different user when pulling the pinned artifacts by specifying credentials in one's .netrc file.

[cheister]

After doing some local testing it looks as if coursier does not use the netrc file so when you are pinning artifacts it will only use the credentials in the coursier credentials.properties file. coursier will also cache the files locally in a path that has the username in the path if it used authentication for the repo. The jvm_external rules just use the same path that coursier returnes when creating the install.json file. I'm not sure there would be a good way to change this without renaming the artifact directory with the username in the name after the call to coursier. This feels like it could be buggy if you happen to have jars from the same repo with and without authentication.

At least one improvement we could look at is adding the credentials in the netrc to the coursier credentials so you would only need to specify them in one place.

[EArbeitman]

Hi,

Just wanted to chyme in on this issue. I also ran into the problem where my username is injected in the maven_install.json file when running pin command:

            {
                "coord": "antlr:antlr:2.7.7",
                "dependencies": [],
                "directDependencies": [],
                "exclusions": [
                    "org.slf4j:slf4j-log4j12"
                ],
                "file": "v1/https/USERNAME%40artifacts.compass-tech.net/repository/maven/antlr/antlr/2.7.7/antlr-2.7.7.jar",
                "mirror_urls": [
                    "https://artifacts.compass-tech.net/repository/maven/antlr/antlr/2.7.7/antlr-2.7.7.jar"
                ],
                "sha256": "88fbda4b912596b9f56e8e12e580cc954bacfb51776ecfddd3e18fc1cf56dc4c",
                "url": "https://[email protected]/repository/maven/antlr/antlr/2.7.7/antlr-2.7.7.jar"
            },