rasterframes
rasterframes copied to clipboard
RasterJoin df.toLayer etc fail on non-standard CRS / Extent col names
This code:
val df = spark.read.raster //....
.select(rf_tile($"red"), rf_extent($"red") as "red_extent", rf_crs($"red") as "red_crs")
.toLayer(tlm)
Fails with error about column crs as crs not found. Pinpointed this to https://github.com/locationtech/rasterframes/blob/develop/core/src/main/scala/org/locationtech/rasterframes/extensions/RasterJoin.scala#L44
I believe in this case since there is no projected raster tile column (note use of rf_tile), it is trying to select crs and extent by name.
Should look for columns by type instead.
A highlight from the stack trace:
at org.apache.spark.sql.Dataset.withColumn(Dataset.scala:2225)
at org.locationtech.rasterframes.extensions.RasterJoin$$anonfun$usePRT$1$3.apply(RasterJoin.scala:46)
at org.locationtech.rasterframes.extensions.RasterJoin$$anonfun$usePRT$1$3.apply(RasterJoin.scala:45)
at scala.Option.map(Option.scala:146)
at org.locationtech.rasterframes.extensions.RasterJoin$.usePRT$1(RasterJoin.scala:45)
at org.locationtech.rasterframes.extensions.RasterJoin$.apply(RasterJoin.scala:52)
at org.locationtech.rasterframes.extensions.ReprojectToLayer$.apply(ReprojectToLayer.scala:52)
at org.locationtech.rasterframes.extensions.DataFrameMethods$class.toLayer(DataFrameMethods.scala:215)