incubator-graphar icon indicating copy to clipboard operation
incubator-graphar copied to clipboard

[Feat][Spark] support of Spark 3.4.x and 3.5.x

Open SemyonSinchenko opened this issue 11 months ago • 7 comments

Is your feature request related to a problem? Please describe. Currently only 3.2.x and 3.3.x versions are supported.

Describe the solution you'd like

  • datasources-34 submodule in spark project
  • datasources-34 profile in top level pom.xml
  • new version in spark CI matrix

Describe alternatives you've considered .

Additional context Existent datasources-32 and datasources-33 may be used as an example.

SemyonSinchenko avatar Feb 27 '24 09:02 SemyonSinchenko

It is a relatively easy task because we have examples of 3.2 and 3.3 submodules in the code. So, I marked it as good first issue.

SemyonSinchenko avatar Feb 27 '24 09:02 SemyonSinchenko

Hi, @SemyonSinchenko, i want to have a try, plz assign to me, thanks!

ShockleysxX avatar Feb 27 '24 17:02 ShockleysxX

@ShockleysxX sure. Let me know if you need some help here.

SemyonSinchenko avatar Feb 27 '24 17:02 SemyonSinchenko

Hi, @ShockleysxX Do you have any progress on this issue, or do you need any help?

acezen avatar May 06 '24 07:05 acezen

The permalink to the related code

  • datasources-34 submodule in spark project this can refer to the datasource-32 and datasource-33
  • datasources-34 profile in top level pom.xml https://github.com/apache/incubator-graphar/blob/f0ae38a81401b9d873a3a8575f83ec5edaef93a1/spark/pom.xml#L37-L81
  • new version in spark CI matrix https://github.com/apache/incubator-graphar/blob/f0ae38a81401b9d873a3a8575f83ec5edaef93a1/.github/workflows/spark.yaml#L45-L52

acezen avatar May 06 '24 07:05 acezen

@ShockleysxX Do you need any kind of help with this issue?

SemyonSinchenko avatar Jun 13 '24 15:06 SemyonSinchenko

can we also add the 3.5.x or even 4.0.x to the roadmap?

xin-hao-awx avatar Aug 07 '24 10:08 xin-hao-awx