deck.gl-layers GeoArrowScatterplotLayer throws 'c.vector is undefined'

As alluded to on Mastodon, I am still having issues with getting GeoArrowScatterplotLayer to work. A reproducible example is below. This seems a very similar issue to what @jaredlander reported here, in fact, I've modified his example in the reprex below. This uses some .arrow data hosted on a CORS enabled public bucket on AWS S3.

<head>
    <meta charset='UTF-8' />
    <title>GeoArrowScatterplotLayer issue</title>
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/Arrow.es2015.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/dist.dev.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@geoarrow/[email protected]/dist/dist.umd.min.js"></script>
    <style>
        map {
            position: absolute;
            top: 0;
            left: 0;
            width: 100vw;
            height: 100vh;
        }

        body {
            font-family: Helvetica, Arial, sans-serif;
            width: 100vw;
            height: 100vh;
            margin: 0;
        }

        #holder {
            border-color: red;
            border-width: 1px;
            border-style: solid;
            position: relative;
            height: 600px;
            width: 95%;
        }
    </style>
</head>

<body>
    <div id="holder"></div>
    <script type="module">
        const geoarrowLayers = window['@geoarrow/deck']['gl-layers'];

        const GEOARROW_POINT_DATA = "https://geoarrow-test.s3.eu-central-1.amazonaws.com/test1_layer.arrow";

        const arrow_table = await Arrow.tableFromIPC(fetch(GEOARROW_POINT_DATA));

        const INITIAL_VIEW_STATE = {
            latitude: 40.72875,
            longitude: 14.00005,
            zoom: 10,
            bearing: 0,
            pitch: 0
        };

        const mainMap = new deck.DeckGL({
            initialViewState: INITIAL_VIEW_STATE,
            id: 'mainMap',
            controller: true,
            container: "holder",
            layers: [
                new geoarrowLayers.GeoArrowScatterplotLayer({
                    id: 'scatter',
                    data: arrow_table,
                    pickable: true,
                    getFillColor: [80, 0, 200],
                    getPosition: d => d.getChild('geometry'),
                    getRadius: 300,
                })
            ]
        });
    </script>
</body>

</html>

This is the error in the console

Any ideas as to what might cause this?

May 24 '24 16:05 tim-salabim

As a general note, I'd suggest testing with the PathLayer before testing with the TripsLayer. The TripsLayer is more complex.

This is probably due to the Arrow name not correctly getting found. We found recently that the name of arrow-js on the page has to be Arrow with a capital A. This was recently fixed in this repo in https://github.com/geoarrow/deck.gl-layers/pull/122 but hasn't been released yet

May 30 '24 10:05 kylebarron

Thanks for the clarification. I saw that PR but assumed it was already released. I'll wait until then.

As a general note, I'd suggest testing with the PathLayer before testing with the TripsLayer. The TripsLayer is more complex.

This example is using ScatterplotLayer. Am I missing something?

May 30 '24 11:05 tim-salabim

Huh... When I copied your code into an HTML file and ran it on localhost, the error traceback has trips-layer in it, which made me confused. Maybe the sourcemaps are incorrect?

May 30 '24 19:05 kylebarron

Strange, if I serve it in firefox via localhost I get

In Chromium I get the same as you

May 31 '24 10:05 tim-salabim

Yeah must be an issue with the source maps...

Jun 02 '24 08:06 kylebarron

I published @geoarrow/[email protected] with the latest main

Jun 02 '24 12:06 kylebarron

Unfortunately, I am still getting the same error...

Jun 02 '24 14:06 tim-salabim

I'm not a JS expert, and personally I always use React with a bundler. Based on the previous issue in https://github.com/geoarrow/deck.gl-layers/issues/109, it seemed like people were able to get something working directly from a CDN.

I don't have a lot of spare time right now, so I'm not able to try and dig in and debug this.

Jun 02 '24 15:06 kylebarron

Maybe @joemarlo can help write CI/CD pipelines for that?

Jun 03 '24 12:06 jaredlander

I'm not a JS expert, and personally I always use React with a bundler. Based on the previous issue in https://github.com/geoarrow/deck.gl-layers/issues/109, it seemed like people were able to get something working directly from a CDN.

@jaredlander did you ever get this working? If so, how?

Jun 24 '24 09:06 tim-salabim

I didn't think we looked at it yet, but I'll remind my folks to take a look.

Jun 24 '24 12:06 jaredlander

Thanks!

Jun 24 '24 12:06 tim-salabim

This does look similar to #109. I don't think we ever got it working straight from a CDN. We cloned the branches, ran npm run build on them, and copied over the static files to the project we were working on. Specifically:

npm run build deck.gl-layers/tree/kyle/bundling-fix branch
npm run build geoarrow-js/tree/kyle/fix-bundling branch
Source both of these new local umd.js files in the head of the example's index.html

Not a clue how to get this on a CDN. Do you have to submit to CDNs or do they just scan and pull the repos' main branch?

It looks like the fix for deck.gl-layers has been merged but not the geoarrow-js fix. So perhaps that is the issue?

Jun 24 '24 19:06 joemarlo

I merged the geoarrow-js pr. If you need me to create a patch release I can

Jun 24 '24 19:06 kylebarron

Thanks, @kylebarron. I think a release is necessary. Do you submit it to npm? I did a little poking around and the CDNs apparently just pull the latest version from there.

Jun 25 '24 12:06 joemarlo

I published 0.3.1 https://github.com/geoarrow/geoarrow-js/releases/tag/v0.3.1

Jun 25 '24 13:06 kylebarron

Thanks everyone! When using the latest versions, we get rid of the JS errors, but we get a webgl warning (this is in Firefox):

WebGL warning: drawArraysInstanced: Instance fetch requires 100, but attribs only supply 0.

<!DOCTYPE html>

<head>
    <meta charset='UTF-8' />
    <title>GeoArrowScatterplotLayer issue</title>
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/Arrow.es2015.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/dist.dev.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@geoarrow/[email protected]/dist/geoarrow.umd.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@geoarrow/[email protected]/dist/dist.umd.min.js"></script>
    <style>
        map {
            position: absolute;
            top: 0;
            left: 0;
            width: 100vw;
            height: 100vh;
        }

        body {
            font-family: Helvetica, Arial, sans-serif;
            width: 100vw;
            height: 100vh;
            margin: 0;
        }

        #holder {
            border-color: red;
            border-width: 1px;
            border-style: solid;
            position: relative;
            height: 600px;
            width: 95%;
        }
    </style>
</head>

<body>
    <div id="holder"></div>
    <script type="module">
        const geoarrowLayers = window['@geoarrow/deck']['gl-layers'];

        const GEOARROW_POINT_DATA = "https://geoarrow-test.s3.eu-central-1.amazonaws.com/test1_layer.arrow";

        const arrow_table = await Arrow.tableFromIPC(fetch(GEOARROW_POINT_DATA));

        const INITIAL_VIEW_STATE = {
            latitude: 40.72875,
            longitude: 14.00005,
            zoom: 10,
            bearing: 0,
            pitch: 0
        };

        const mainMap = new deck.DeckGL({
            initialViewState: INITIAL_VIEW_STATE,
            id: 'mainMap',
            controller: true,
            container: "holder",
            layers: [
                new geoarrowLayers.GeoArrowScatterplotLayer({
                    id: 'scatter',
                    data: arrow_table,
                    pickable: true,
                    getFillColor: [80, 0, 200],
                    getPosition: arrow_table.getChild('geometry'),
                    getRadius: 30,
                    radiusMinPixels: 3
                })
            ]
        });
    </script>
</body>

</html>

Jun 28 '24 11:06 tim-salabim

WebGL warning: drawArraysInstanced: Instance fetch requires 100, but attribs only supply 0.

That could be a bug in deck.gl v9

Jun 28 '24 13:06 kylebarron

@felixpalmer this is the issue we just talked about in the morning session of sdsl2024. Maybe you can have a look at what the problem could be with the above example?

Sep 18 '24 11:09 tim-salabim

It's best if you don't use the minified scripts, so go with:

<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/dist.dev.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@geoarrow/[email protected]/dist/geoarrow.umd.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@geoarrow/[email protected]/dist/dist.umd.js"></script>

With this in place I tried to debug, but something seems off with your Arrow file, I tried loading it in https://arrowviewer.pages.dev/ and it also gives an error:

Abort(): Could not load arrow column of type `struct`

Could you try with another arrow file?

At any rate the issue seems to be with the Arrow handling rather than anything related to the v9 migration

Sep 18 '24 12:09 felixpalmer

Yeah this is a "separated" GeoArrow column, which deck.gl-layers does not yet support. See https://github.com/geoarrow/deck.gl-layers/issues/3

Sep 18 '24 21:09 kylebarron

Great success! With the help of @paleolimbot I was able to write interleaved coordinates and the example now works:

Thanks for all your patience, everyone!

Sep 26 '24 09:09 tim-salabim