Choropleth: allow to use custom maps

[kravets-levko]

What type of PR is this? (check all applicable)

• [x] Feature

Description

Allow user to type arbitrary GeoJSON URL; existing build-it maps are available as dropdown options. Also, now instead of setting "Key Type" (type of key field from query result) user should map query result field to one of GeoJSON field.

Also fixed few small bugs in visualization.

Related Tickets & Documents

https://discuss.redash.io/t/choropleth-map-per-regions/4437

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

[kravets-levko]

The only thing that really makes this feature completely useless is this:

@arikfr Do you have any ideas what we can do with it?

[arikfr]

@kravets-levko not sure 🤔 The simplest solution? Give up on dynamic values there and keep adding different maps to our build. We should just make it very simple and documented process, so anyone can make a PR for this.

[kravets-levko]

@arikfr How about adding connect-src *; to default CSP rules? According to docs, it will allow only requests triggered by JS code (and it fixes the issue mentioned in my first comment - I already tried it on my dev instance).

[kravets-levko]

Everything works fine, except of that CSP issue :point_up:

[arikfr]

@arikfr How about adding connect-src *; to default CSP rules? According to docs, it will allow only requests triggered by JS code (and it fixes the issue mentioned in my first comment - I already tried it on my dev instance).

Allowing requests to be triggered by JS code sounds like a potential risk, and the kind of risk CSP supposed to prevent.

I thought of a different solution: have a Proxy endpoint in Redash that loads this JSON endpoint and returns it. The complex part is to make this endpoint secure by checking that this URL is indeed in the visualization definition.

[arikfr]

The complex part is to make this endpoint secure by checking that this URL is indeed in the visualization definition.

But this actually doesn't have to be that hard:

1. Make a call to /api/resource-proxy with params: URL to load & visualization id.
2. In the handler, load the visualization, check that it actually defines the URL.

Only issue: what to do before saving? :|

[kravets-levko]

Proxy is an option of course.

Only issue: what to do before saving? :|

What if proxy instead of checking visualization object will just allow to load any JSON files? Even if we'll need such a feature in other visualization (maybe one day?) - most likely we'll not need anything except of JSON. So proxy may load URL, check if it returned JSON and then pass that JSON to frontend. And of course allow only GET methods. Does it sound safe?

[arikfr]

Thinking of it; making only GET requests should be fine.

[kravets-levko]

@arikfr I added a basic implementation of proxy to load custom maps in https://github.com/getredash/redash/pull/4599/commits/813d97a62ca256327f3dd317d54b5f91a9b41f7e Most likely it has to be improved, so please review it - I'll appreciate any suggestions about it :bowing_man:

[kravets-levko]

Another relevant idea to discuss (and possibly implement): here I added some more metadata to Choropleth maps, but having it in our codebase will make it harder to add custom maps. The idea is to embed those metadata into GeoJSON files (it's allowed by GeoJSON specs). It will solve problem with custom maps, but also there are few cons: 1) map name couldn't be embedded because it's needed before the file is loaded, and 2) it will be a bit harder to find those mappings (e.g. to fix them).

[jimsparkman]

Is this still being considered? Would love to see this feature added.

[kravets-levko]

Hi @jimsparkman! Yes, I'll resume my work on it soon. I have to re-think some details first

[opioom]

@kravets-levko are u able to estimate any deadline? We use redash for our clients (big hospitals in PL) and we need to analyze the patients diagnosis' distribution. This would be a great feature for us to have it in redash. Currently the client has to speak with ESRI which of course cost a lot of money.

[kravets-levko]

Hi @opioom! Sorry, no any estimates for now. I totally understand how useful this feature is, but ATM I'm loaded with other stuff.

[dengc367]

great job. when to merge to the master branch?

[chrispruitt]

Nice work on this! Any update yet on when this will be merged?

[opioom]

it's been 6 months since my last post so....do we have any progress? Any other options like workaround that could help to me visualize the PL regions with any data?

[mathieubossaert]

it's been 6 months since my last post so....do we have any progress? Any other options like workaround that could help to me visualize the PL regions with any data?

Waiting for this great feature too, for the moment I "dirty" replaced map files (countries.geo.json and japan.prefecture.geo.json) by files with same structure and names but with my own data (square grid and municipalities).

I created specific volumes for it.

    volumes:
      - /opt/redash/maps/countries.geo.json:/app/client/dist/data/0962607.countries.geo.json
      - /opt/redash/maps/japan.prefectures.geo.json:/app/client/dist/data/650f3ee.japan.prefectures.geo.json

[edouardjmn]

@mathieubossaert Thank you so much! Works well 🙌🏻

[opioom]

@mathieubossaert that's sounds like a solution! Do you have any detailed guide how to do that? I desperate need that function for healthcare analysis.

[mathieubossaert]

Hi @opioom , I will document this trick / workaround after the release of v10.

[mathieubossaert]

@opioom As promised on september. We "simply" need to change the content of the 3 geo.json files (keeping same names) used by redash, with your own content. In my case I needed to show 3 kind of maps :

• cities and villages from Occitanie
• 10km square mesh.
• french "Departements"

So I replace Japan prefectures, counties and USA geojson files with mine.. Let's say my geo.json files are located on the host into /opt/redash/maps/ directory. Their names are

• communes_occitanie.geo.json
• mailles_10km.geo.json
• departements.geo.json

We "simply" need to mount server container geojson files to the hosted mines. In the end of the server section of the docker-compose file, add

    volumes:
      - /opt/redash/maps/communes_occitanie.geo.json:/app/client/dist/data/0962607.countries.geo.json
      - /opt/redash/maps/mailles_10km.geo.json:/app/client/dist/data/dac34f3.japan.prefectures.geo.json
      - /opt/redash/maps/departements.geo.json:/app/client/dist/data/eed73d1.usa-albers.geo.json

Before that, you can check the name of the files used by redash container (name changes for a file on V10 upgrade on my instance)

docker exec -t -i redash_server_1 ls /app/client/dist/data/ 

0962607.countries.geo.json
eed73d1.usa-albers.geo.json dac34f3.japan.prefectures.geo.json

[opioom]

Thanks a lot @mathieubossaert!! Will try to follow your guide on my test env. Edit: I've made a small modification because it didn't work via docker compose. I simply replaced json files inside redash_server (docker cp) with my own json files. The only fix I have to make are missing labels on a map, but it's probably related to the json file structure.

[mathewtrivett]

Realise this is an old PR. I was wondering whether it is possible to customise the polygons / geojson used by the choropleth visualisation? Would be happy to contribute to get this integrated.

[guidopetri]

@kravets-levko , thanks for the PR! We've updated a lot of things now that we're Community-driven so - if you're still interested (or @mathewtrivett ) in getting this merged - would you mind rebasing off master to re-run the CI, as well as updating merge conflicts?

We're trying to clean up our PR todo list, so if you're not interested, that's fine - we'll close the PR in about a week if we don't hear back. If you're interested in reopening the PR afterwards, we would also very much welcome that.

[justinclift]

Lets keep this one around for a while longer. It'll probably need to be reworked, but it's too important to just lose to history in the short term. :smile: