Collect HTTP Headers from Coursier for authentication

[markvandertol]

This PR lets Scala Steward also pick up custom headers used for authentication. This is for example needed for Gitlab. I think looking at csrConfiguration.value.authenticationByRepositoryId is the cleanest and most generic way to achieve this.

Example snippet of a .sbt file from a project that sets a custom HTTP-header for authentication with Coursier:

resolvers += "gitlab-repo" at "https://gitlab.example.com/api/v4/groups/1/-/packages/maven",
csrConfiguration ~= _.addRepositoryAuthentication("gitlab-repo", Authentication(Seq( ("private-token", "xxx") )))

It works for SBT and Mill, but doesn't work for Maven

[codecov[bot]]

Codecov Report

Merging #2533 (ffd36e5) into main (fcd5ea3) will decrease coverage by 0.67%. The diff coverage is 34.09%.

@@            Coverage Diff             @@
##             main    #2533      +/-   ##
==========================================
- Coverage   81.32%   80.65%   -0.68%     
==========================================
  Files         146      146              
  Lines        2592     2621      +29     
  Branches       43       45       +2     
==========================================
+ Hits         2108     2114       +6     
- Misses        484      507      +23     

Impacted Files	Coverage Δ
.../scala/org/scalasteward/core/application/Cli.scala	100.00% <ø> (ø)
...a/org/scalasteward/mill/plugin/StewardPlugin.scala	0.00% <0.00%> (ø)
...la/org/scalasteward/sbt/plugin/StewardPlugin.scala	0.00% <0.00%> (ø)
...a/org/scalasteward/core/coursier/CoursierAlg.scala	71.11% <35.71%> (-14.19%)	:arrow_down:
.../org/scalasteward/core/buildtool/mill/parser.scala	66.66% <50.00%> (-1.52%)	:arrow_down:
...org/scalasteward/core/buildtool/maven/parser.scala	100.00% <100.00%> (ø)
...in/scala/org/scalasteward/core/data/Resolver.scala	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[markvandertol]

Thanks for having a look. I replaced the Option[List[Header]] with List[Header]. That indeed simplifies the code. I feel that encoding { "key": "foo", "value": "bar" } as object instead of array is clearer, the same as the credentials object is encoded as { "user": "foo", "pass", "bar" }. Also, because you could have more than one header. If you want I can change this.

[julienrf]

May I kindly request a review from the maintainers?

[cipriansofronia]

@markvandertol, could u also update the gitlab docs on how we can use this new feature?

[markvandertol]

@markvandertol, could u also update the gitlab docs on how we can use this new feature?

@cipriansofronia I've added a few lines explaining how to authenticate against a Gitlab repository. Does this clarify enough how it works? No specific code for Scala Steward is needed.

[jchapuis]

Looking forward to this one being released, thanks a lot @markvandertol

[cipriansofronia]

@fthomas can this PR get some ❤️ from you? 😇

[markvandertol]

Is there anything preventing this PR from being merged? 🙂

[exoego]

@fthomas Do you have a time to test this with your test Steward instance ??

[jchapuis]

👋 is there anything holding up the merge at this point?

[kaffepanna]

wave is there anything holding up the merge at this point?

I am wondering that to. This would really be a good thing to have when managing dependencies in private gilab repos

[987Nabil]

wave is there anything holding up the merge at this point?

I am wondering that to. This would really be a good thing to have when managing dependencies in private gilab repos

It is not only gitlb. Also Google Cloud Storage or Artifactregistry or managing packages in AWS need this feature.

[exoego]

@markvandertol Sorry for waiting so long. Could you resolve the conflict? 🙇

[markvandertol]

No worries. I resolved the merge conflict. 🙂

[michaltomanski]

Thanks Mark for this PR! It looks promising.

Any chance to have it merged some time soon?

[fthomas]

Thanks for having a look. I replaced the Option[List[Header]] with List[Header]. That indeed simplifies the code.

Having the fields optional allows to read in persisted RepoCaches of previous Scala Steward runs. If the header fields are non-optional, Scala Steward fails to read these files:

[info] 2022-07-14 14:48:58,983 INFO  ──────────── Steward scala-steward-org/scala-steward ────────────
[info] 2022-07-14 14:48:58,983 INFO  Check cache of scala-steward-org/scala-steward
[info] 2022-07-14 14:48:59,080 ERROR Failed to parse or decode JSON from /home/frank/data/code/scala-steward/core/workspace/store/repo_cache/v1/github/scala-steward-org/scala-steward/repo_cache.json
[info] io.circe.DecodingFailure$$anon$2: Attempt to decode value on failed cursor: DownField(headers),DownField(MavenRepository),DownArray,DownField(resolvers),DownArray,DownField(dependencyInfos)

Another alternative would be to bump the version of the repoCacheStore here. But for this small change I would prefer making the fields optional.

Other than that, this PR looks good to me.

[markvandertol]

Thanks @fthomas for having a look at the PR.

I fixed the issue by using a default value of Nil for the headers, and using deriveConfiguredCodec as shown below.

  @annotation.nowarn("cat=unused-locals")
  implicit val resolverCodec: Codec[Resolver] = {
    implicit val customConfig: Configuration = Configuration.default.withDefaults
    deriveConfiguredCodec
  }

I had to suppress the warning, as the implicit is only used by a macro. I could also have used the compiler flag -Ywarn-macros:after, but that would be inconsistent with other @annotation.nowarn usages.

[exoego]

Sorry for the late response.

[jchapuis]

Any chance this can get shipped in a new release? 😃 🙏 thanks!

[fthomas]

Just noticed that this change broke StewardPlugin.scala for old sbt versions. See #2847 for details.