zeppelin icon indicating copy to clipboard operation
zeppelin copied to clipboard

Published 20 hours ago • verified publisher icon apache

ZEPPELIN-3551. Upgrade Scala to 2.11.12

Open zjffdu opened this issue 6 years ago • 16 comments

What is this PR for?

This is Just to update scala to 2.11.12 which to be consistent with spark (SPARK-24418)

What type of PR is it?

[Improvement]

Todos

  • [ ] - Task

What is the Jira issue?

  • https://issues.apache.org/jira/browse/ZEPPELIN-3551

How should this be tested?

  • CI pass

Screenshots (if appropriate)

Questions:

  • Does the licenses files need update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

zjffdu avatar Jun 19 '18 07:06 zjffdu

@Leemoonsoo @felixcheung Could you help review it ? Thanks

zjffdu avatar Jun 21 '18 08:06 zjffdu

Will merge it if no more comments

zjffdu avatar Jun 26 '18 00:06 zjffdu

LGTM

Leemoonsoo avatar Jun 26 '18 01:06 Leemoonsoo

specifically in SPARK-24418 - there code changes needed but not release yet (Spark 2.4.0?)

felixcheung avatar Jun 26 '18 06:06 felixcheung

I tested it against spark master branch, and it works. What kind of problems does spark have on 2.11.12 ? I notice SPARK-24418 is already merged.

zjffdu avatar Jun 26 '18 06:06 zjffdu

something in SparkILoop. given that I don't think we can/should upgrade scala broadly - it might only work for Spark 2.4.0 (unreleased) but not older/current releases

felixcheung avatar Jun 26 '18 06:06 felixcheung

Thanks @felixcheung , I will revert the change in the root pom file and only update scala version in spark module.

zjffdu avatar Jun 26 '18 06:06 zjffdu

? it looks like spark.version is 2.2.0. Or under profile, 2.3.0 as far as I know SPARK-24418 is fixed only in (unreleased) master branch

felixcheung avatar Jun 27 '18 06:06 felixcheung

It is to fine to compile with spark 2.2, the main change is on SparkScala211Interpreter.scala which is due to scala repl api changes in scala 2.11.12.

zjffdu avatar Jun 27 '18 08:06 zjffdu

Perhaps you could elaborate on why we should do this now since Spark doesn’t actually support it? Is there a reason that I have missed?

felixcheung avatar Jun 27 '18 16:06 felixcheung

This is for the next release of zeppelin which I suppose should supported the latest spark 2.4.0 which would use scala 2.11.12. Just want to catch up with spark.

zjffdu avatar Jun 28 '18 01:06 zjffdu

well, ok. though might be hard to tell - we are juggling multiple releases 2.1.3, 2.2.2, 2.3.2 - might take some time to get to 2.4.0.

felixcheung avatar Jun 28 '18 06:06 felixcheung

Anyway, this PR is just for the next release of spark that supports scala 2.11.12, I can hold it if you have any concern

zjffdu avatar Jun 28 '18 08:06 zjffdu

Hi, @zjffdu , @felixcheung , @Leemoonsoo .

Since Spark 2.1.3/2.2.2/2.3.2 is out and Spark 2.4.0 RC3 is started, can we restart this?

dongjoon-hyun avatar Oct 11 '18 08:10 dongjoon-hyun

Hey @zjffdu, busy? I can take this over - looks there's only minor conflict.

HyukjinKwon avatar Dec 04 '18 09:12 HyukjinKwon

Thanks @HyukjinKwon feel free to take over this

zjffdu avatar Dec 04 '18 10:12 zjffdu