rasterframes icon indicating copy to clipboard operation
rasterframes copied to clipboard

RasterJoin df.toLayer etc fail on non-standard CRS / Extent col names

Open vpipkt opened this issue 5 years ago • 1 comments

This code:

val df = spark.read.raster //....
          .select(rf_tile($"red"), rf_extent($"red") as "red_extent", rf_crs($"red") as "red_crs")
          .toLayer(tlm)

Fails with error about column crs as crs not found. Pinpointed this to https://github.com/locationtech/rasterframes/blob/develop/core/src/main/scala/org/locationtech/rasterframes/extensions/RasterJoin.scala#L44

I believe in this case since there is no projected raster tile column (note use of rf_tile), it is trying to select crs and extent by name.

Should look for columns by type instead.

vpipkt avatar Sep 10 '20 20:09 vpipkt

A highlight from the stack trace:

	at org.apache.spark.sql.Dataset.withColumn(Dataset.scala:2225)
	at org.locationtech.rasterframes.extensions.RasterJoin$$anonfun$usePRT$1$3.apply(RasterJoin.scala:46)
	at org.locationtech.rasterframes.extensions.RasterJoin$$anonfun$usePRT$1$3.apply(RasterJoin.scala:45)
	at scala.Option.map(Option.scala:146)
	at org.locationtech.rasterframes.extensions.RasterJoin$.usePRT$1(RasterJoin.scala:45)
	at org.locationtech.rasterframes.extensions.RasterJoin$.apply(RasterJoin.scala:52)
	at org.locationtech.rasterframes.extensions.ReprojectToLayer$.apply(ReprojectToLayer.scala:52)
	at org.locationtech.rasterframes.extensions.DataFrameMethods$class.toLayer(DataFrameMethods.scala:215)

vpipkt avatar Sep 10 '20 20:09 vpipkt