Missing addProtocol method fot custom protocols, Protomaps / pmtiles support

[punov]

trafficstars

Motivation

At the moment, we have a frontend integration with Protomaps vector tiles, that have a full support of Maplibre GL: https://protomaps.com/docs/frontends/maplibre

Inside our Mobile application, we use Maplibre as tool for rendering maps, but we don't have a chance to render same vector tiles since this custom protocol is not supported out of the box, and there's no way to add it.

Implementation

Expectations is to have the same addProtocol method to provide a loadFn for custom tile servers like pmtiles.

import * as pmtiles from "pmtiles";

let protocol = new pmtiles.Protocol();
maplibregl.addProtocol("pmtiles",protocol.tile);

If there's any known workaround to provide support for custom protocol using the existing Maplibre API, I'd appreciate any directions / advices.

[jthiard]

Hi @punov I didn't got deep into it, but if I understand it right the addProtocol method in maplibre-gl-js is used to override with javascript code the tiles fetching when using a specific protocol in source url.

As this maplibre-react-native lib is a wrapper on top of maplibre-gl-native - and not maplibre-gl-js, fetching the tiles is made at the native level, and you won't be able to customize it with javascript code.

I'm unsure if it may be customized at the android / iOS level (with java/objectiveC or kotlin/swift hooks) or if it requires to go deeper at the C++ level.

[punov]

@jthiard thanks for your answer, after going through the source code, I have the same understanding. I'm ok to customize the native code to support it, but at the moment I struggle finding the proper hook for tileserver fetch operations.

addProtocol method is as simple as calling the function to fetch the data according to current coordinates and zoom level, so getting closer to the function that executes it in native code is a step #1 for me.

[lseelenbinder]

@punov You'll be looking for functionality around here:

src/mbgl/storage

Most likely an adaptation of src/mbgl/storage/http_file_source.hpp and platform/default/src/mbgl/storage/http_file_source.cpp with some of platform/default/src/mbgl/storage/file_source_manager.cpp.

It's a bit of a mess, but generally you can register handlers for types of files, and there's a default FileSourceManager normally invoked. You'll need to create your own and pass it through via ResourceOptions: src/mbgl/map/map.cpp.

Does this help point you in the right direction?

[lpatrun]

Is there any progress on this? Or some instructions on how to do it?

[KiwiKilian]

PMTiles is currently beeing worked on at the MapLibre Native side https://github.com/maplibre/maplibre-native/pull/2882.

[lpatrun]

Is there any plan for the near/distant future to have an addProtocol functionality like it is implemented for the "maplibre-gl"?

[KiwiKilian]

AFAIK there is nobody working on addProtocol, you can try implementing something like this as described by https://github.com/maplibre/maplibre-react-native/issues/28#issuecomment-1419505405 – but I can't yet imagine a clean way where this would be supported by MapLibre RN.

PMTiles support is much more likely, once it's available in MapLibre Native.

[KiwiKilian]

Would anyone need an addProtocol method for anything else then PMTiles? Would love to hear other use cases, if there are any.

[tyrauber]

Wouldn't addProtocol be required to support an api_key on styleJSON?

What about the maplibre-contour plugin which uses addProtocol under the covers to accomplish the terrain layers from a raster-dem?

[onizukanaim]

My solution meanwhile : Thank you use dom

"use dom";

import maplibregl from "maplibre-gl";
import { Protocol } from "pmtiles";
import layers from "protomaps-themes-base";
import { useEffect } from "react";
import { Map } from "react-map-gl";
import { View } from "react-native";
import { useColorScheme } from "~/utils/useColorScheme";

export default function MapViewWeb() {
  const { colorScheme } = useColorScheme();
  useEffect(() => {
    let protocol = new Protocol();
    maplibregl.addProtocol("pmtiles", protocol.tile);
    return () => {
      maplibregl.removeProtocol("pmtiles");
    };
  }, []);

  return (
    <Map
      attributionControl={false}
      style={{ width: "100vw", height: "100vh" }}
      mapStyle={{
        glyphs:
          "https://protomaps.github.io/basemaps-assets/fonts/{fontstack}/{range}.pbf",
        sprite:
          "https://api.protomaps.com/styles/v4/light/en.json?key=NON_COMMERCIAL_USE_KEY",
        version: 8,
        sources: {
          sample: {
            type: "vector",
            url: "pmtiles://https://world.pmtiles",
          },
        },
        layers: layers(
          "sample",
          colorScheme,
          "en"
        ) as mapboxgl.LayerSpecification[],
      }}
      mapLib={maplibregl as any}
    />
  );
}

[conbrad]

PMTiles support was recently merged in maplibre-native: https://github.com/maplibre/maplibre-native/pull/2882

I think they just cut a release, can we bump that dependency here?

[KiwiKilian]

Sure, it's on my radar, but please have a little patience – It's not even yet released/deployed on both platforms.

Once #589 is merged, anyone can control the underlaying MapLibre Native versions quite easily. Which would allow everyone to test a new native release, once it's available, without changes to this repository.

[KiwiKilian]

I still think adding addProtocol is out of scope for this project, it belongs towards MapLibre Native. And MapLibre Native seems to solve these requirement differently, as we see pmtiles:// getting an integrated solution.

Quoting @tyrauber

Wouldn't addProtocol be required to support an api_key on styleJSON?

What about the maplibre-contour plugin which uses addProtocol under the covers to accomplish the terrain layers from a raster-dem?

An apiKey should just be added to the URLs as parameters, see Stadia Maps docs for example.

For contour lines there is an open design proposal in the spec.

So basically, this is mainly always a requirement towards MapLibre Native. If there is ever some addProtocol for MapLibre Native, we will for sure try to expose this! Therefore I'm planning to close this issue, once we have PMTiles support through bumping the native dependencies. Feel free to comment, if you see other use cases, but otherwise I recommend to research your use case and propose such solutions towards MapLibre Native, so we can implement on top of it.

[tyrauber]

Totally agree, @KiwiKilian, I have no problem letting the Native team do the heavy lifting.

I believe the primary issue with the apiKey is that some map tile styles are hosted by a third-party, and adding a apiKey to the style json url doesn't automatically add it to the tile or asset requests. Perhaps we should create an issue and track that separately.

I would very much like to see Michael's contour lines proposal get implemented. His JS plugin which I think implements this is very nice.

[github-actions[bot]]

:tada: This issue has been resolved in version 10.0.0-beta.19 :tada:

The release is available on:

• npm package (@beta dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[KiwiKilian]

I'm closing this one, as PMTiles support is available.

Supporting API keys already has it's own issue #424.

If anyone has another use case for addProtocol feel free to chime in, or go forward towards MapLibre Native.

[abdelhameedhamdy]

I'm closing this one, as PMTiles support is available.

Supporting API keys already has it's own issue #424.

If anyone has another use case for addProtocol feel free to chime in, or go forward towards MapLibre Native.

I was waiting for supporting PMTiles protocol in maplibre-rn, is it available in the latest beta release, if so, is there a guide for using it ?

[conbrad]

I'm closing this one, as PMTiles support is available. Supporting API keys already has it's own issue #424. If anyone has another use case for addProtocol feel free to chime in, or go forward towards MapLibre Native.

I was waiting for supporting PMTiles protocol in maplibre-rn, is it available in the latest beta release, if so, is there a guide for using it ?

The new versions of maplibre-native that contains PMTiles support were bumped and merged here in the beta branch here it looks like (awesome, thank you!): https://github.com/maplibre/maplibre-react-native/releases/tag/v10.0.0-beta.19

You can reference the maplibre-native docs to use PMTiles: https://maplibre.org/maplibre-native/android/examples/data/PMTiles/

[abdelhameedhamdy]

I'm closing this one, as PMTiles support is available. Supporting API keys already has it's own issue #424. If anyone has another use case for addProtocol feel free to chime in, or go forward towards MapLibre Native.

I was waiting for supporting PMTiles protocol in maplibre-rn, is it available in the latest beta release, if so, is there a guide for using it ?

The new versions of maplibre-native that contains PMTiles support were bumped and merged here in the beta branch here it looks like (awesome, thank you!): https://github.com/maplibre/maplibre-react-native/releases/tag/v10.0.0-beta.19

You can reference the maplibre-native docs to use PMTiles: https://maplibre.org/maplibre-native/android/examples/data/PMTiles/

Aha, thank you for clarifying that, I thought it was exposed in v10.0.0-beta.19.

[KiwiKilian]

It should be as easy as using a pmtiles:// url for a VectorSource or within your map style:

<VectorSource
  id="pm-tiles"
  url="pmtiles://https://example.com/your-tiles.pmtiles"
>
  <FillLayer
    id="fill"
    sourceLayerID="some-source-layer-in-your-pmtiles"
    style={{ fillColor: "red" }}
  />
</VectorSource>

We have an open issue to better document the available protocols #592.