leaflet-providers icon indicating copy to clipboard operation
leaflet-providers copied to clipboard

Splitting out the list of providers in a standalone JSON file

Open SylvainCorlay opened this issue 2 years ago • 12 comments

Currently, the available providers are listed in a JavaScript object that is defined inline in the source code of leaflet-providers.

It would be really useful to make that information available directly in a JSON file that could be used from other projects. For example, xyzservices extracts that information from leaflet-provider.js to make the same information available to Python. Xyzservices has some adoption in the scientific Python ecosystem (Geopandas, ipyleaflet, contextily, leafmap, bokeh, and others) - meaning that this project has in fact a lot of indirect adoption...

This separate JSON file could even be published independently of the leaflet-providers JS source, which would probably make it valuable for other users.

SylvainCorlay avatar Jun 10 '22 17:06 SylvainCorlay

Hi @SylvainCorlay nice idea, we just have to find a way to generate automagically this JSON file from the repo. Do you have any hint for that ? @jieter any thoughts on this ?

brunob avatar Jun 13 '22 09:06 brunob

Hi @brunob, we already have that magic in xyzservices. We scrape your JS to get that JSON but the result is slightly adapted to work within our use case. But the Python code for that is here https://github.com/geopandas/xyzservices/blob/main/provider_sources/_parse_leaflet_providers.py and a resulting JSON here https://github.com/geopandas/xyzservices/blob/main/provider_sources/leaflet-providers-parsed.json.

martinfleis avatar Jun 13 '22 09:06 martinfleis

@martinfleis do you think we can use your script on our repo with a github ci or anything like that ?

brunob avatar Jun 13 '22 09:06 brunob

@brunob Yeah, we do that within this GHA https://github.com/geopandas/xyzservices/blob/main/.github/workflows/update_providers.yaml using this makefile

martinfleis avatar Jun 13 '22 10:06 martinfleis

Nice, let's hope that @jieter could look into it or propose an alternative method :)

brunob avatar Jun 13 '22 10:06 brunob

We could generate this JSON as a one-off, include the JSON in your repository, and then read the JSON from your JavaScript instead of having the object hard coded in the code base.

This would make it a single source of truth.

SylvainCorlay avatar Jun 14 '22 17:06 SylvainCorlay

Yes, it would be nice to maintain the list in a machine readable format, and generate leaflet-providers.js from it + a template. I am not sure about JSON though. In the current definitions, we use some comments: (1 2 3 4), which I think is a valuable capability not possible in classic json.

Other options would be JSONC, but that would require consumers of the file to use non-standard parsers too.

Any suggestions?

jieter avatar Jun 14 '22 18:06 jieter

What about using YAML instead if we need comments? That feels as a bit more standard solution than JSONC.

martinfleis avatar Jun 14 '22 18:06 martinfleis

The big advantage of JSON is that you can parse it from either Python or JS without a third-party dependency.

SylvainCorlay avatar Jun 15 '22 20:06 SylvainCorlay

+1 for JSON too

brunob avatar Jun 16 '22 08:06 brunob

One option to keep comments in the JSON without using non-standard parsers would be to include an optional note or comment field with a string that is currently in the comment and ignore it while parsing the JSON to show it on your website or within xyzservices.

martinfleis avatar Jul 13 '23 19:07 martinfleis

We would also like this at TileProviders.jl (which is used in Leaflet.jl but is generalised so everyone can use it)

Currently we pull json from geopandas xyzservices because its easier to parse

To minimise downstream workload I suggest that rather than YAML or a new JSON format we use the JSON format xyzservices has already defined.

rafaqz avatar Apr 03 '24 19:04 rafaqz