[Bug] MVTLayer layer breaks when used with the DataFilterExtension

[hydroes]

Description

When applying filters the filtering is not consistent and when applying a filter and removing it, and then re-applying it the data points on the map can change. Filtering does not work as expected.

Flavors

• [X] Script tag
• [X] React
• [ ] Python/Jupyter notebook
• [ ] MapboxOverlay
• [ ] GoogleMapsOverlay
• [ ] CartoLayer
• [ ] ArcGIS

Expected Behavior

Applying the same filters with the same data should render the same data points but they change and jump around even though the point co-ordinates change.

Steps to Reproduce

Our configuration:

1. Data filter: new DataFilterExtension({ filterSize: 4, categorySize: 1 }

2. getFilterValue: function getFilterValue(f: { properties }) { return [ new Date(f.properties.date_of_transfer).getTime() / 1000, Number(f.properties.total_floor_area) || 0, Number(f.properties.estimated_bedrooms_min) || 0, Number(f.properties.tenure === "F"), ]; }

3. Filter range: const filterRange = [ [fromDate, toDate], // these are numbers [filters?.floorArea?.min || 0, filters?.floorArea?.max || Math.pow(2, 32) - 1], [filters?.bedrooms?.min || 0, filters?.bedrooms?.max || Math.pow(2, 32) - 1], [filters?.tenure !== "F" ? 0 : 1, filters?.tenure === "L" ? 0 : 1], ];

4. sample data props that are being filtered by: "properties": { "date_of_transfer": "2015-09-01", "total_floor_area": "121.00", "estimated_bedrooms_min": 3, "tenure": "F" }

5. render subLayer props: renderSubLayers(props) { return [ new GeoJsonLayer({ ...props, stroked: false, filled: true, pointType: "circle+text", getTextColor: [0, 0, 0, 255], textOutlineColor: [255, 255, 255, 255], getText: textAccessors[unitPreference][priceMode], getTextAngle: 0, getTextBorderWidth: 2, getTextAnchor: "middle", getTextPixelOffset: [0, -16], textSizeScale: 1, textSizeUnits: "pixels", textCharacterSet: characterSet, textFontFamily: "Helvetica Neue, Helvetica, Arial, sans-serif", textOutlineWidth: 5, getTextSize: 14, lineWidthUnits: "pixels", lineWidthMaxPixels: 1, lineWidthMinPixels: 1, lineWidthScale: 1, pointRadiusUnits: "pixels", getPointRadius: 12, pointRadiusMaxPixels: 12, getFillColor: [132, 204, 21, 255], getLineColor: [255, 255, 255, 255], getLineWidth: 2, textFontSettings: { sdf: true, fontSize: 32, buffer: 8, }, updateTriggers: { getFillColor: [priceMode, distribution], getText: [unitPreference, priceMode], }, }), ]; },

Data side note: We are using the binary true with mvt data.

The issue: The dots on the map do not filter correctly and 'jump' around.

Please let us know if you need further info

Environment

• Framework version: 9.0.12
• Browser: Chrome
• OS: MacOs

Logs

No response

[Pessimistress]

Hi, thanks for reporting a bug. Here are some things you can do to help a maintainer look into your issue:

• Format your code so it's readable
• Provide a fully functional reproduction of your issue in Codepen/Code Sandbox/etc
• If not possible, a dataset that can be used for testing
• If not possible, screenshots/video that demonstrate your issue

[landtechjoe]

This should provide an example of how to recreate the issue using an edited version of the cloudpen example for the scatterplot layer.

Steps to reproduce

• check the second checkbox item
• check the first checkbox item

Expected outcome Stations with either Street or Avenue in the name would be displayed

Actual outcome Some of the stations visible with just the second checkbox selected become hidden, some new ones become visible. Some stations are only visible when both these checkboxes are selected but not when either one of them are selected

I am hoping I've just made a naive error in how I am doing it and you can point me in the direction of how to fix it :)

const { DeckGL, ScatterplotLayer, DataFilterExtension } = deck;

/*
 * https://deck.gl/docs/api-reference/layers/scatterplot-layer
 */
const roadTypes = ["Street", "Avenue", "Blvd.", "Road", "Airport", "Way", "Dr.", "St."];
const form = document.createElement("form");
form.onSubmit = (e) => e.preventDefault();
form.style.cssText = "position: absolute; top: 0; left: 0; z-index: 50;";
form.addEventListener("change", (e) => {
  const selectedRoadTypes = Array.from(e.target.form.elements)
    .filter((checkbox) => checkbox.checked)
    .map((checkbox) => checkbox.value);
  const newLayer = layer.clone({
    filterCategories: selectedRoadTypes,
  });
  console.log(`Updating filterCategories to`, selectedRoadTypes);
  deckInstance.setProps({
    layers: [newLayer],
  });
});

roadTypes.forEach((addressPart) => {
  const label = document.createElement("label");
  const el = document.createElement("input");
  label.textContent = addressPart;
  el.type = "checkbox";
  el.id = addressPart;
  el.value = addressPart;
  label.appendChild(el);
  form.appendChild(label);
});
document.body.appendChild(form);


const layer = new ScatterplotLayer({
  id: "ScatterplotLayer",
  data: "https://raw.githubusercontent.com/visgl/deck.gl-data/master/website/bart-stations.json",

  /* props from ScatterplotLayer class */

  // antialiasing: true,
  // billboard: false,
  // filled: true,
  getFillColor: [255, 140, 0],
  getLineColor: [0, 0, 0],
  // getLineWidth: 1,
  getPosition: (d) => d.coordinates,
  getRadius: (d) => Math.sqrt(d.exits),
  // lineWidthMaxPixels: Number.MAX_SAFE_INTEGER,
  lineWidthMinPixels: 1,
  // lineWidthScale: 1,
  // lineWidthUnits: 'meters',
  radiusMaxPixels: 100,
  radiusMinPixels: 1,
  radiusScale: 6,
  // radiusUnits: 'meters',
  stroked: true,
  extensions: [new DataFilterExtension({ categorySize: 1 })],
  getFilterCategory: (properties) => {
    for (const roadType of roadTypes) {
      if (properties.address.includes(roadType)) {
        return roadType;
      }
    }
    return "";
  },
  filterCategories: roadTypes,

  /* props inherited from Layer class */

  // autoHighlight: false,
  // coordinateOrigin: [0, 0, 0],
  // coordinateSystem: COORDINATE_SYSTEM.LNGLAT,
  // highlightColor: [0, 0, 128, 128],
  // modelMatrix: null,
  opacity: 0.8,
  pickable: true,
  // visible: true,
  // wrapLongitude: false,
});

const deckInstance = new DeckGL({
  mapStyle: "https://basemaps.cartocdn.com/gl/positron-gl-style/style.json",
  initialViewState: {
    longitude: -122.4,
    latitude: 37.74,
    zoom: 11,
    maxZoom: 20,
    pitch: 30,
    bearing: 0,
  },
  controller: true,
  getTooltip: ({ object }) =>
    object &&
    `${object.name}
${object.address}`,
  layers: [layer],
});

[Pessimistress]

I cannot reproduce your issue on my box. codepen

If you are still experiencing it with the latest release, please share a screen recording of what you see.

[landtechjoe]

https://github.com/visgl/deck.gl/assets/97101845/18600318-2f99-4a9b-9197-86444fc33811

You can see in this video. I select the first box and some of the items are hidden. When I select the next box it should just show a few more items but for some reason a bunch of them disappear and a few appear. There doesn't seem to be any correlation between the selected options and the items rendered on the map. I would expect a few more points to appear when the additional checkbox is selected and none should disappear.

There is only one address in the demo data that has the street abbreviation "St." If I only check the "St." checkbox it shows pretty much everything. I can't see any pattern in the behaviour that would point to a cause. The "Road" checkbox seems to be the only that works and shows only those points having an address with "Road" in them.

[landtechjoe]

Just nudging this if you can find time to look at this issue again. Thanks

[Pessimistress]

@landtechjoe unfortunately I cannot reproduce the issue. What model is your Mac and what GPU does it have?

[landtechjoe]

Okay, I've passed this around to some other people who have access to different machines and it looks like their Windows machines are fine, x86 Macs (tested on 2017 Mac book pro) are also fine but M1 mac book pros seem to suffer from this bug. Currently trying to find someone with an M2 mac to see if the same problem is for all ARM chips on Mac or just the M1. It works fine on my phone which is ARM so I guess this is just limited to ARM macs.

[landtechjoe]

Had a couple of other people report back that this problem also presents on iPhone's which also use an ARM chip so definitely looks to be related specifically to Apple's ARM architectures. Is there anything we can provide in addition to this that would help. One of guys I work with suggested providing uniforms that are being generated on affected and unaffected machines. I could get this information for you if it would be useful for you

[TylerMatteo]

I'm not sure if I was experiencing exactly this same issue but we had a problem wherein DataFilterExtension wasn't working as expected with MVTLayer on M2 MBPs but was working on M1s and generally working on x86 machines. Upgrading from v9.0.27 to v9.1.1 seems to have resolved the issues, I'm guessing because it includes #9350