rasterframes

rasterframes copied to clipboard

So far, I have seen one difference. The pandas `set_option` function throws on passing in a -1 as is done in the custom dataframe formatter in `rf_ipython` submodule. It should...

vpipkt

spark.read.geotrellisCatalog do not use spark.hadoop.fs.s3a.* config correctly

6

## Current situation When I to read Geotrellis catalog with an `s3a://` URI using `spark.read.geotrellisCatalog`, I get the following error: https://gist.github.com/jdenisgiguere/61161a1bd9636ec91c3b75cbb6a845b9 A workaround is to first read data with the...

jdenisgiguere

Column name conflict when using raster_join on data that was already raster joined.

4

``` Py4JJavaError: An error occurred while calling o59.rasterJoin. : org.apache.spark.sql.AnalysisException: Reference 'spatial_index_agg' is ambiguous, could be: spatial_index_agg, spatial_index_agg.; ``` Easy to reproduce, just try to `raster_join` 3 rasters. On second...

mjgolebiewski

This one is up for debate. We could possibly trim down the pyrasterframes `setup_requires` list by relying on the/a `requirements.txt` to define the dependencies for building docs. And carefully check...

vpipkt

Avoid having to explicitly call `st_extent` when the area covered by a tile/raster is defined as geometry. Use case is when user converts the `extent` to a `geometry` so that...

metasim

Now that #417 is in, would like to do a similar thing with GeoJson reader.

vpipkt

Expression would need 5 arguments 😜 Register as SQL function. See note here about SQL https://github.com/locationtech/rasterframes/blob/develop/docs/src/main/paradox/reference.md#rf_mask_by_bits

vpipkt

Something like `rasterframes-all_`

metasim

RasterJoin should use `st_overlaps` not `st_intersects`

2

In raster join we use a DataFrame join to gather rows from both dataframes in order to construct the joined tiles. In the process of doing this currently [uses `st_intersects`](https://github.com/locationtech/rasterframes/blob/develop/core/src/main/scala/org/locationtech/rasterframes/extensions/RasterJoin.scala#L41)....

vpipkt

Enable pyarrow de/serialization for Tile

5

Given the following correct looking code in an environment with `pyarrow` installed: ```python from scipy.stats import kurtosis from pyspark.sql.functions import pandas_udf, PandasUDFType @pandas_udf('double', PandasUDFType.SCALAR) def tile_kurtosis(t): return kurtosis(t.cells, axis=None) spark.read.raster(path)...

vpipkt