gravitino icon indicating copy to clipboard operation
gravitino copied to clipboard

Published 20 hours ago • verified publisher icon apache

[Subtask] [spark-connector] support multi spark&scala version

Open FANNG1 opened this issue 1 year ago • 3 comments

Describe the subtask

support spark 2.4 and scala 2.13 for now, we should support multi spark and scala versions

  • [x] #3007
  • [ ] #3305
  • [x] #3330
  • [ ] #3320

Parent issue

#1227

FANNG1 avatar Jan 17 '24 09:01 FANNG1

Hi @FANNG1 will we consider support for spark3.1 and spark3.3 latter? Currently the main versions of spark in XIAOMI are spark3.1 and spark3.3.

caican00 avatar Mar 14 '24 01:03 caican00

yes, but we planned to do this after the basic feature of the spark-connector is set up. If you're interested, you could do some work.

FANNG1 avatar Mar 14 '24 01:03 FANNG1

yes, but we planned to do this after the basic feature of the spark-connector is set up. If you're interested, you could do some work.

It sounds reasonable. Let's set up the basic feature first and if there are version compatibility issues, we can build on a lower version first in XIAOMI.

caican00 avatar Mar 14 '24 03:03 caican00