rasterframes icon indicating copy to clipboard operation
rasterframes copied to clipboard

Spark 3.5: Fix StaticInvoke constructor differences in ManualTypedEncoder; add tests; fixes #630

Open AnudeepKonaboina opened this issue 3 months ago • 2 comments

Problem: Spark 3.5 changed org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke constructor shape, breaking reflective construction used by encoders.

Solution: Build StaticInvoke reflectively by selecting the max-arity constructor (8/9 args) and trying both target class and companion module (static forwarder) for apply. Fallback to NewInstance when no static forwarder matches arity. Compatible with Spark 3.4 and 3.5.

Added: ManualTypedEncoderSpec ensuring case-class decoding via static apply works through ManualTypedEncoder.staticInvoke.

Fixes: https://github.com/locationtech/rasterframes/issues/630.

AnudeepKonaboina avatar Oct 17 '25 04:10 AnudeepKonaboina

@pomadchin ! This is the new PR with ECA signed .. the old one #631 is closed as the PR was raised from a different account.Also this was tested by a couple of users and it worked without throwing the NoSuchMethod Error

AnudeepKonaboina avatar Oct 17 '25 05:10 AnudeepKonaboina

Hey @pomadchin ! Could you please review this and approve of the changes look good

AnudeepKonaboina avatar Nov 25 '25 12:11 AnudeepKonaboina