openrouteservice icon indicating copy to clipboard operation
openrouteservice copied to clipboard

Add support for newer population data

Open rabidllama opened this issue 4 years ago • 2 comments

The population data used in openrouteservice isochrones is currently the 2015 data from JRC (https://data.jrc.ec.europa.eu/dataset/jrc-ghsl-ghs_pop_gpw4_globe_r2015a). There is newer version of the data available that was released in 2019 (https://data.jrc.ec.europa.eu/dataset/0c6b9751-a71f-4062-830b-43c9f432370f/resource/fb9565ed-9eff-426f-91a8-3f32afb525b9) which should be used, though it only has data up until 2015 (there are some enhancements that have been made to the dataset since the initial 2015 release).

Also, the current implementation uses PostGIS functions that are deprecated in newer versions (ST_SummaryStatistics). Support should be added to openrouteservice so that the preferred function (ST_SummaryStatisticsAgg) is available for use when the data is stored in newer PostGIS versions.

rabidllama avatar Nov 17 '20 08:11 rabidllama

The data was updated to 2019 dataset (with 2015 data). Also Postgres and Postigs for the database were updated.

Neither ST_SummaryStatistics nor ST_SummaryStatisticsAgg exist anymore. ST_SummaryStats is the current available function which is already used. (ST_SummaryStatsAgg existsbut doesn't seem to give much benefit).

Currenlty GHSL data is present in a custom added SRID 954009 while it should be the native 54009 that comes with postgis.

Instead of hardcoding the SRID in https://github.com/GIScience/openrouteservice/blob/master/openrouteservice/src/main/java/org/heigit/ors/isochrones/statistics/postgresql/PostgresSQLStatisticsProvider.java#L123-L130

The srid can be extracted, from the table, which would allow switching to data with the correct SRID:

try {
    // extract the srid from the geometry column
    int mollweide_srid = 0;
    ResultSet rs = dataSource.getConnection().
            prepareStatement("SELECT ST_srid(" + geomColumn + ") FROM " + tableName + " LIMIT 1").
            executeQuery();
    if (rs.next()){
        mollweide_srid = rs.getInt("st_srid");
    }
    String sql = null;
    for (String property : properties) {
        String polyGeom = isochrone.getGeometry().toText();
        if ("total_pop".equals(property)) {
            sql = "SELECT ROUND(SUM((ST_SummaryStats(ST_Clip(" +
                    geomColumn + ", poly))).sum)) AS total_pop FROM " +
                    tableName + ", ST_Transform(ST_GeomFromText('" + polyGeom +
                    "', 4326), "+ mollweide_srid + ") AS poly WHERE ST_Intersects(poly, " +
                    geomColumn + ") GROUP BY poly;";
        }
    }

TheGreatRefrigerator avatar Sep 09 '21 08:09 TheGreatRefrigerator

In the meantime there is a new data set from 2022 with data for 2020: https://ghsl.jrc.ec.europa.eu/ghs_pop2022.php

TheGreatRefrigerator avatar Mar 07 '23 15:03 TheGreatRefrigerator