GeoFabrics icon indicating copy to clipboard operation
GeoFabrics copied to clipboard

General Performance improvements related to geometry manipulations

Open rosepearson opened this issue 3 years ago • 2 comments

There are various options to explore to reduce the setup time (i.e. queuing all the function calls before beginning to execute them):

  1. Clip the tile_index_extents to the region to rasterise buffered by the radius before the for loop - will limit the number of tiles to search through in 'select_lidar_files'. Do this in the roughness _add_tiled_lidar_chunked method as well image Addressed as part of https://github.com/rosepearson/GeoFabrics/issues/131
  2. In _define_chunk_region explore buffering and clipping the shapely primatives instead of creating geopadas.GeoDataFrame's before the final step image

rosepearson avatar Dec 19 '22 21:12 rosepearson

Note this built on the work in https://github.com/rosepearson/GeoFabrics/issues/127

rosepearson avatar Dec 19 '22 21:12 rosepearson

Also look at: Not quite the same subject - but when picking this up again could look at using dask with pandas to remove the horrendous bottle neck that exists when calculating the 'open waterway elevations'. See https://github.com/rosepearson/GeoFabrics/blob/main/src/geofabrics/processor.py#L2386

Each row should be able to operate in parallel

rosepearson avatar Jun 19 '23 03:06 rosepearson

Have looked at these options. Dask GeoPandas wasn't quite ready with apply, and have addressed some of the define chunk region.

rosepearson avatar Jan 14 '25 23:01 rosepearson