pydeck: Loading a local .glb file in ScenegraphLayer

[harisbal]

Discussed in https://github.com/visgl/deck.gl/discussions/6323

^{Originally posted by harisbal October 25, 2021} At the moment pydeck cannot load a local .glb file to use in the ScenegraphLayer

Assuming that a BoxAnimated.glb file exists in the folder, the following does not work:

import pydeck as pdk
layer = pdk.Layer(
    "ScenegraphLayer",
    data=[{'coordinates': [-122.466233, 37.684638]}],
    scenegraph=r'./BoxAnimated.glb',
    get_position="coordinates",
)

view_state = pdk.ViewState(latitude=37.684638, longitude=-122.466233, zoom=18, bearing=0, pitch=0)

r = pdk.Deck(layer, initial_view_state=view_state, map_style="light")
r.show()

[kylebarron]

I assume glb is a binary format? We could load it on the Python side, serialize it literally to JS as we do with Numpy arrays, then use that ArrayBuffer on the JS side?

[Pessimistress]

@kylebarron GLB is the binary format of GLTF. I believe pydeck has implemented a mechanism to transport binary data. We have an experimental feature on the JavaScript side to resolve data locally:

deck._addResources({my_model: <uint8array>}); // a resource is arbitrary content

const layer = new ScenegraphLayer({
  ...
  scenegraph: 'deck://my_model' // this will resolve to the above content
});
deck.setProps({layers: [layer]});

[kylebarron]

Thanks for the background; I've never worked with 3D models myself.

I believe pydeck has implemented a mechanism to transport binary data.

Yes, in https://github.com/visgl/deck.gl/blob/master/bindings/pydeck/pydeck/data_utils/binary_transfer.py we have implementations for serializing a Numpy array to a binary format, and then passing that binary data over Jupyter's "Comm" Mechanism to the JS side. So we'd just need to adjust _prepare_binary_data to handle other data objects than Pandas DataFrames. In this case, we'd just need to accept the binary data blob given by the user.

[harisbal]

Hello. Is there any possibility to add this feature in the near future?