Will Mleap support Scala 2.13?

[masi-ghodrati]

We have planned to use Mleap in our framework. Our framework uses Scala 2.13. We can't downgrade to 2.12, as we use some of the new features. I would like to know if you plan on upgrading to Scala 2.13 in the near future? ? And if there is any documentation/resource that can help us upgrade to the right Scala version and make Combust compatible with our framework?

[jsleight]

We are on spark 3.2 which is cross compatible for scala 2.12 and 2.13. If you (or anyone) wants to submit a PR to cross compile mleap for both scala 2.12 and 2.13, then I'm happy to review it and get it merged.

Skimming through the scala migration guide it seems the biggest changes are w.r.t. some collections APIs that I suspect mleap doesn't use too much.

[masi-ghodrati]

thank you for your prompt response.I'm gonna work on that.thanks again

[masi-ghodrati]

There are some dependencies which are not supported by Scala 2.13, such as xgboost4j. It's a blocker for updating the entire modules to Scala 2.13. Since our framework only uses the runtime module, do you think we could update it to 2.13 and make it compatible with our framework?

[jsleight]

Ah, that makes sense. In principle I think you can make only some of the modules be cross-compiled with scala 2.13 if you tweaked https://github.com/combust/mleap/blob/master/project/MleapProject.scala. Right nowthe build settings are all shared from Common.buildSettings, but if you want to make a new Common.buildSettingsNoScala2_13 and use that for the xgboost modules then I think that is ok for now.

[masi-ghodrati]

thank you

[nicster]

It seems that xgboost4j now added Scala 2.13 support: https://github.com/dmlc/xgboost/pull/9099. It isn't released yet though, but something to keep an eye on.

[dotbg]

This issue was one of the motivators to open that PR :) We can test things with the snapshot version.

"XGBoost4J Snapshot Repo" at "https://s3-us-west-2.amazonaws.com/xgboost-maven-repo/snapshot/" 

I'll try to ask whether there are plans on making a new xgboost release

[scheruku-in]

Hi @jsleight We are using Spark 3.3 and invoke the below mleap modules in our project:

bundle-ml
mleap-base
mleap-core
mleap-runtime

We plan to upgrade to Scala 2.13 , so I would like to know if the Scala 2.13 support is going to be added for mleap?

Thanks

[jsleight]

#860 Is working on it.

[scheruku-in]

Thanks @jsleight Is there any ETA on when the 2.13 build will be available?

[jsleight]

Don't know, could ask on that PR.

[dotbg]

In PR 864 I've took slightly different approach, ensuring that all other mleap dependencies support scala 2.13. I will consider going forward after that MR is reviewed.

[juliangp]

@dotbg do you have an estimate for when you will have a Scala 2.13 version released?

[dotbg]

I'd love to be able to do that, but I am too busy at my current job at this moment, sorry.

[juliangp]

@dotbg is there something that we can do to help?

[jsleight]

#864 is merged so in theory someone can make the project cross compile now without needing to worry about the dependencies being compatible. If anyone wants to start a PR, I'm happy to review and get things merged.

[prashil1996]

@jsleight I need to use mleap with scala 2.13 due to some CVEs in my project and the same is not possible. Any idea when https://github.com/combust/mleap/pull/860 will be merged?

[jsleight]

Not sure, but once it has a clean build I'm happy to merge it. I see you've commented on that PR as well, which is good.

[dotbg]

Blocked by https://github.com/dmlc/xgboost/issues/9765