sbt-typelevel icon indicating copy to clipboard operation
sbt-typelevel copied to clipboard

Project verification job

Open rossabaker opened this issue 3 years ago • 3 comments

Half-baked idea:

*githubWorkflowCheck runs everywhere, but doesn't vary by JVM or Scala version

  • This is also true of scalafmtSbtCheck.
  • It would also be true of prospective validation of scala-steward.conf

Should there be a project job for the project-level verifications, so they're run only once? They aren't that slow, so it's not a priority, but getting these wrong fails lots of jobs right now.

rossabaker avatar Jan 23 '22 05:01 rossabaker

Oh, I see. I was thinking about this too wrt the scalafmtSbt check. I'd feel funny skipping the githubWorkflowCheck since it's sort of a self-consistency thing.

To take this a step further: what I have seen some projects do is put all their formatting/linting into a separate job from tests. MiMa could be separate too maybe. It buys some additional parallelism and shaves a slice off the primary job matrix. Worth considering?

It's also worth noting that a significant slow down comes from the changes in https://github.com/typelevel/sbt-typelevel/pull/65. Starting sbt from scratch for each step tacks on quite a bit of overhead. I'd love to use sbt's server/client mode here and spent some time looking into it, but it has various issues e.g. https://github.com/djspiewak/sbt-github-actions/issues/74. No good ideas for this.

armanbilge avatar Jan 23 '22 06:01 armanbilge

Yeah, I remembered that SBT startup being faster, but the http4s build with its 80000 keys or whatever takes like 30 seconds to reload on GHA. Those separate steps may have been a mistake.

I think of the scalafmt and header checks as per-crossbuild. Those need to run across the Scala and Platform (but probably not Java) matrix. Those don't require compilation, so they might do well as a single job.

I don't think we'd want to move MiMa (or for builds that use them, semantic scalafixes), because those require compilation. Those steps could probably be reduced to a primary Java, but probably part of the main compilation and test flow.

rossabaker avatar Jan 23 '22 15:01 rossabaker

Those separate steps may have been a mistake.

I still think the trade-off is worth it, in terms of compositionality.

Regarding MiMa and scalafixes, they do require compilation, but they don't require fastOptJS or a rather large test suite. So there is sort of a balance here. We could also change the order in which they run.

That's a good point about the Java axis though. We can make these steps conditional so that they run only on the primary Java. It's a small win for now, but an easy one.

armanbilge avatar Jan 23 '22 16:01 armanbilge