Runtime issue running with spark 1.6

[INRIX-Trang-Nguyen]

Could you tell me if magellan is compatible with spark 1.6?

I am running into this issue trying to convert a parsed RDD to DF:

case class Trip(trip_id: String, probe_id: String, provider_id: String, start_time: Date, start_hour: Integer, start_point: Point, trip_mean_speed: Double, trip_distance_m: Double ) import sqlContext.implicits._ val tripDF = tripSource.loadMovingTrips(processingDay, sourceUrl).toDF()

Error: Exception in thread "main" java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaUniverse$JavaMirror;

Thanks, Trang

[harsha2010]

@INRIX-Trang-Nguyen , magellan is not compatible with Spark 1.6 I am working on a release that is compatible with Spark 1.6 but since Spark 1.6 introduces a lot of changes that affect Magellan;s performance, it is turning out to be a slower upgrade than i anticipated.. look for a release in the next month

[INRIX-Trang-Nguyen]

Thanks. Is it still possible to use 1.6 with just rdds, or will that break as well?

Trang

From: Ram Sriharsha [mailto:[email protected]] Sent: Friday, February 12, 2016 12:05 PM To: harsha2010/magellan [email protected] Cc: Trang Nguyen [email protected] Subject: Re: [magellan] Runtime issue running with spark 1.6 (#53)

@INRIX-Trang-Nguyenhttps://github.com/INRIX-Trang-Nguyen , magellan is not compatible with Spark 1.6 I am working on a release that is compatible with Spark 1.6 but since Spark 1.6 introduces a lot of changes that affect Magellan;s performance, it is turning out to be a slower upgrade than i anticipated.. look for a release in the next month

— Reply to this email directly or view it on GitHubhttps://github.com/harsha2010/magellan/issues/53#issuecomment-183469468.

[INRIX-Trang-Nguyen]

Also, does it work with 1.5?

From: Trang Nguyen Sent: Friday, February 12, 2016 12:53 PM To: 'harsha2010/magellan' [email protected]; harsha2010/magellan [email protected] Subject: RE: [magellan] Runtime issue running with spark 1.6 (#53)

Thanks. Is it still possible to use 1.6 with just rdds, or will that break as well?

Trang

From: Ram Sriharsha [mailto:[email protected]] Sent: Friday, February 12, 2016 12:05 PM To: harsha2010/magellan <[email protected]mailto:[email protected]> Cc: Trang Nguyen <[email protected]mailto:[email protected]> Subject: Re: [magellan] Runtime issue running with spark 1.6 (#53)

@INRIX-Trang-Nguyenhttps://github.com/INRIX-Trang-Nguyen , magellan is not compatible with Spark 1.6 I am working on a release that is compatible with Spark 1.6 but since Spark 1.6 introduces a lot of changes that affect Magellan;s performance, it is turning out to be a slower upgrade than i anticipated.. look for a release in the next month

— Reply to this email directly or view it on GitHubhttps://github.com/harsha2010/magellan/issues/53#issuecomment-183469468.

[cris-pin]

Hi there harsha2010, I wondered if you had got any further with 1.6 support and just wanted to say thank-you for such excellent work so far! Kind regards