Cross building

[ittaiz]

So this isn't anything urgent but popped into my head and thought it would be best to start the conversation now before 2.12 is released and we might have more pressure. 2 questions:

1. How to parameterize the rule with respect to scala version? (don't think it's complicated but haven't looked into it).
2. How to cross build a target to both 2.11 and 2.12 like sbt supports. This is useful for both rolling out the upgrade in an organization and for OSS libraries.

[johnynek]

I think this is similar to #14

I agree we should work on it.

The ideal case is that we just point at different repositories and maybe some functions to get the particular version numbers out.

We can certainly factor common code to something like scala_common.bzl and then call that from scala2_11.bzl and scala2_12.bzl etc...

I don't think it is too hard, I just have not taken a stab.

Probably scala2_10.bzl might be the first to do since many still use scala 2.10 and there are many libraries published that we could check with.

[johnynek]

this could be a useful approach: https://github.com/bazelbuild/rules_scala/pull/84

[ittaiz]

Indeed. I think one of the other rules uses it (rust? Go?) On יום ד׳, 10 באוג׳ 2016 at 23:35 P. Oscar Boykin [email protected] wrote:

this could be a useful approach: #84 https://github.com/bazelbuild/rules_scala/pull/84

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/80#issuecomment-238995491, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUIF6Ypb3faPQFhyvpabhposk4aWA0bks5qejYZgaJpZM4JTZ4n .

[pauldraper]

https://github.com/bazelbuild/rules_scala/pull/84 doesn't let you cross build, right? It just gives you a way to globally change the version?

A couple more thoughts

1. Cross-building (as opposed to just choosing one version) is very important. a. Most popular Scala libraries choose cross-build against two or more versions of Scala, so that they can be more widely consumed. b. Even if a project is not for pubic consumption, it can take some time to migrate a large codebase and dependencies to a new Scala version. I used SBT + https://github.com/lucidsoftware/sbt-cross to cross and and migrate incrementally during 2.10 -> 2.11, and it was very helpful.
2. The cross-built versions project can differ in more than just Scala version, e.g. some 2.11 projects need scala-xml, while in 2.10, this dependency was not separate.
3. It would be nice to keep just the one worker process, and use classloaders to load the different Scala compilers.

[johnynek]

I agree cross-building is somewhat interesting, but it is a very low priority for me personally. I'm happy to review PRs.

Bazel is generally used for private (logical) monorepos. There is no support for publishing, for instance. We could have multiple parallel builds, or we could say that the cross building solution is to have your CI bump the scala version you are using.

At Twitter, where we used the similar pants build system, there was no support for this, and in general keeping a large monorepo green with more than one version of scala, I think will be very challenging, not to mention tooling to support versioning the external jars correctly, something bazel does not understand (that scala has multiple parallel artifacts for each different release of scala).

[oscar-stripe]

see #251

[KaoruDev]

Checking in if this is something that will be supported. Otherwise, in order for us to adopt Bazel, we'll have to get all of our services on the same Scala version (which isn't ideal).

[johnynek]

I think this thread is still the state of the art. You can switch scala versions and so you can manage to have your CI set two different versions, but having a two versions in the same build is not yet supported.

It sounds like you want to build some parts of the code with only version A and others with one version B. Is that right?

A key challenge is this: we want to support interoperability with java (java can depend on scala). But we can’t add two versions of the scala library on the class path. So we need some way to error if that happens.

I don’t think we have found a design where multiple versions of scala in the same build in bazel works great.

[KaoruDev]

SGTM, thanks for the tip on the CI switch

[harpocrates]

I wonder what Databricks did internally to support cross_scala_library from this post. That looks exactly like what I was looking for, but based on this thread it doesn't look like this is supported here and I couldn't find any of their code in the open.

[eed3si9n]

Wrote a blog post cross build anything with Bazel documenting the stateful cross building (aka switch at CI approach).