spark-druid-connector
spark-druid-connector copied to clipboard
Update some codes for spark version 2.4.4
When I tested original code with spark(2.4.4), I got some trouble. So, I updated 4 difference steps.
- Change spark, scala version and dependency modules version in
build.sbt
. - Add case to check field like
Alias(Cast(....))
- Add case to check LogicalPlan in scanPlan.
- before
- df.select('fieldA').take(1) --> checked
case Project(projectList, _)
- df.take(1) --> checked
case _
- df.select('fieldA').take(1) --> checked
- after
- df.select('fieldA').take(1) --> checked
case Project(projectList, _)
- df.take(1) --> checked
case a if a.isInstanceOf[LogicalPlan]
- df.select('fieldA').take(1) --> checked
- Change log level when the druid client requests some thing to druid server.