GeoIO.jl icon indicating copy to clipboard operation
GeoIO.jl copied to clipboard

Native support for GeoPackage file format

Open juliohm opened this issue 6 months ago • 13 comments

Issue

The GeoPackage file format was developed by OGC a while ago to store "vector" and "raster" data as SQLite containers. It is widely adopted in open datasets like GADM and GeoBR.

GeoIO.jl currently relies on the external GDAL library to load/save .gpkg files:

load

https://github.com/JuliaEarth/GeoIO.jl/blob/e2f1bc5e9d6b79f9ce5892b4e143fe2cbd260003/src/extra/gis.jl#L66-L69

save

https://github.com/JuliaEarth/GeoIO.jl/blob/e2f1bc5e9d6b79f9ce5892b4e143fe2cbd260003/src/extra/gis.jl#L53-L55

Given the numerous reports of GDAL build failures (at least one per year for the last 5 years):

  • https://github.com/JuliaGeo/GDAL.jl/issues/191
  • https://github.com/JuliaGeo/GDAL.jl/issues/179
  • https://github.com/JuliaGeo/GDAL.jl/issues/174
  • https://github.com/JuliaGeo/GDAL.jl/issues/173
  • https://github.com/JuliaGeo/GDAL.jl/issues/158
  • https://github.com/JuliaGeo/GDAL.jl/issues/151
  • https://github.com/JuliaGeo/GDAL.jl/issues/130
  • https://github.com/JuliaGeo/GDAL.jl/issues/126
  • https://github.com/JuliaGeo/GDAL.jl/issues/113
  • https://github.com/JuliaGeo/GDAL.jl/issues/101

we can't rely on it moving forward.

Bounty

We need a new extra/gpkg.jl implementation that handles GeoPackage "vector" tables in native Julia.

The load implementation should consist of SQLite.jl queries that convert

  1. the table of attributes stored in the database into a Tables.jl table
  2. the geometries (with CRS information) into a vector of geometries geoms from Meshes.jl

followed by a geotable = georef(table, geoms) call to combine these two objects into a GeoTable.

The save implementation should do the reverse operation, i.e. it should save

  1. the values(geotable) as a SQLite attribute table
  2. the domain(geotable) as SQLite geometries

in the final GeoPackage file.

A solution that

  1. implements the corresponding OGC GeoPackage specification
  2. can load/save all *.gpkg files from GeoArtifacts.GADM
  3. can load/save all *.gpkg files from GeoArtifacts.GeoBR
  4. adheres to our code style and passes our current tests

will get a /bounty $500.

Tips

  • The GeoPackage guidance document contains useful information, including an implementation guide
  • If you need any assistance, please feel free to ask questions in our Zulip channel

juliohm avatar May 27 '25 20:05 juliohm

💎 $500 bounty • JuliaEarth

Steps to solve:

  1. Start working: Comment /attempt #167 with your implementation plan
  2. Submit work: Create a pull request including /claim #167 in the PR body to claim the bounty
  3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

  • If anything is unclear, ask for clarification here or in our Zulip channel
  • Please read our GDSJL book if you need an overview of our ecosystem
  • Make sure your PR follows the surrounding code style and is idiomatic
  • Low quality AI PRs will not receive review and will be closed

Thank you for contributing to JuliaEarth/GeoIO.jl!

Attempt Started (UTC) Solution Actions
🟢 @MAVRICK-1 Sep 03, 2025, 04:24:58 PM #172 Reward

algora-pbc[bot] avatar May 27 '25 20:05 algora-pbc[bot]

I'd be happy to continue working with you @juliohm

So if GeoArtifacts provide gpkg, is there example user or package which produces GeoTables that need to be saved as gpkg. Ie. Does anyone produce a GeoTable without reading it from a gpkg? Maybe a better question would be: for the user of gpkg = save(geotable), where did they acquire the geotable information from?

Omar-Elrefaei avatar May 27 '25 20:05 Omar-Elrefaei

Hi @Omar-Elrefaei , that is great to hear.

Answering your question: any GeoTable over a GeometrySet where the geometries are described in terms of 2D coordinates can be saved into gpkg. Like other GIS formats, the gpkg format also has some partial support for 3D coordinates.

Does that answer your question? A common thing to do in practice is load *.shp files and save into *.gpkg because the former has limitations, see http://switchfromshapefile.org

juliohm avatar May 27 '25 20:05 juliohm

  • To claim a bounty, you need to provide a short demo video of your changes in your pull request
  • Do not ask to be assigned unless you've contributed before

Please ignore these bot messages. I need to find where I can configure them later.

juliohm avatar May 27 '25 21:05 juliohm

Hi, @juliohm I'm planning to give this bounty a go since I have graduated recently. However, I'm from Mongolia and algora account doesn't support my country. So I don't know about how to get the bounty in an event of completion. Although I'm not an expert, I have some knowledge about GIS, Julia and Python. I'm able to commit lot of hours that's it.

MendeBadra avatar Jun 10 '25 06:06 MendeBadra

@MendeBadra feel free to take a jab at it. I'm currently a bit occupied and I'm not fully sure when am I going to be able to work on it.

Omar-Elrefaei avatar Jun 10 '25 11:06 Omar-Elrefaei

@MendeBadra it is unfortunate that Algora doesn't support Mongolia yet. Maybe try to contact the Algora team to see why that is the case and if there is any upcoming support.

juliohm avatar Jun 10 '25 12:06 juliohm

Ok, I am going to attempt implementing it. I will try to contact Algora team.

MendeBadra avatar Jun 10 '25 12:06 MendeBadra

Currently, I am not working on this issue. However, I am learning SQL at the moment. I might be little bit unqualified for this task.

MendeBadra avatar Jun 17 '25 07:06 MendeBadra

/attempt #167

MAVRICK-1 avatar Jul 16 '25 21:07 MAVRICK-1

#174 Just pushing my work I have collected so far.

Not interested in cash reward, just want to contribute to simply be able to showcase accolades on my github profile

Definitely interested in future contributions, so please accept this as an entry contribution for future issues

jph6366 avatar Aug 04 '25 13:08 jph6366

@juliohm is this open ? i saw #177 . i can help to complete it

Sahelisaha04 avatar Sep 11 '25 03:09 Sahelisaha04

There is another ongoing PR. Let's wait a bit more before parallel attempts.

Em qua., 10 de set. de 2025, 22:32, Saheli Saha @.***> escreveu:

Sahelisaha04 left a comment (JuliaEarth/GeoIO.jl#167) https://github.com/JuliaEarth/GeoIO.jl/issues/167#issuecomment-3277253259

@juliohm https://github.com/juliohm is this open ? i saw #177 https://github.com/JuliaEarth/GeoIO.jl/pull/177 . i can help to complete it

— Reply to this email directly, view it on GitHub https://github.com/JuliaEarth/GeoIO.jl/issues/167#issuecomment-3277253259, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZQW3LS5GO4I27V524MCNL3SDUMFAVCNFSM6AAAAAB6A7S3Z6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENZXGI2TGMRVHE . You are receiving this because you were mentioned.Message ID: @.***>

juliohm avatar Sep 11 '25 03:09 juliohm

@jph6366 finished the reader part of this issue in #188. We merged the PR today.

He already has an open PR for the writer in #182, which we will hopefully update and merge soon.

Thank you all for the comments and interest on this issue.

juliohm avatar Dec 18 '25 12:12 juliohm