Height support for foxglove.Grid

[jtbandes]

The foxglove.Grid type currently allows custom coloring based on any of its fields. It would be nice to also support varying the height (z-position) of the grid's vertices based on one of the fields. This could take the form of an item in settings to select a field to use as the height value. If none is selected it would default to the current behavior of z=0.

An example implementation of a height tile rendering implementation is available in https://github.com/foxglove/studio/pull/2555.

There is an open question about how height values should be interpreted, and we will need to do some research about how other libraries/frameworks handle this issue, and document our resolution to it.

Does the height at cell i,j specify the height of the upper-left corner of the rendered cell, or its centroid, or what?

For example, if a grid is 2x2 and contains these height values:

grid.data["height"]:
+---+---+
| 0 | 1 |
+---+---+
| 2 | 3 |
+---+---+

To render this as a 2x2 grid of squares, the renderer needs to create a geometry that has 3 vertices along each edge of the grid. This means that the grid's data doesn't contain enough information to decide on the height of all the vertices.

• Assuming the height refers to the upper-left corner of each cell, we don't know the height of the last row/column of vertices:

  0---1---?
  | / | / |
  2---3---?
  | / | / |
  ?---?---?
  

• Assuming the height refers to the center of each cell, we don't know the heights of vertices on the outer edges, and might need strange interpolation for inner vertices:

  ?---0---?
  | / | / |
  0---?---3
  | / | / |
  ?---3---?
  

• ...or we could use 8 instead of 2 triangles per cell, allowing us to set center heights exactly but still leaving ambiguity at the edges:

  … = interpolate value
  ?---?---?
  |\|/|\|/|
  ?-0-…-1-?
  |/|\|/|\|
  ?-…-…-…-?
  |\|/|\|/|
  ?-2-…-3-?
  |/|\|/|\|
  ?---?---?
  

• Or do we come up with a way to specify the extra row/column of data in the grid itself?

[jhurliman]

Since this will commonly be used for elevation heightmaps, I suggest following the GeoTIFF specification in RasterPixelIsPoint mode.

[jtbandes]

From my reading that means the user's data would need to be N+1 x N+1?

Pixel-is-point standard with DTED, where a 3" data set has 1201x1201points. The edge columns and rows are shared with the neighboring cells.

The PixelIsPoint raster grid space R uses the same coordinate axis names as used in PixelIsArea Raster space, with increasing I to the right, increasing J down. The first pixel-value however, is realized as a point value located at $(0,0)$. An $N$ by $M$ pixel image consists of points which fill the mathematically defined bounds $(0,0),(N-1,M-1)$.

So would we require the user to change their column_count and row_stride, and change the cell colors into vertex colors (and ignore the last row/column) when height is enabled, or would there be other fields we need to add to Grid to encode this somehow?

[jhurliman]

change the cell colors into vertex colors

You're right that there's a discrepancy between storing PixelIsArea data like color and PixelIsPoint elevation data in the same data array. Should we have a separate data array for height?

[jtbandes]

Not just a separate data array, but a separate list of fields (or just require it to be a single float32 field?) And at that point it's almost like a whole separate Grid topic 😕

I'm actually not sure if it makes sense to store both PixelIsArea and PixelIsPoint data in the same grid. Are there any legitimate use cases where someone would want to do that? We could add a pixel_is_point field to the grid and default to false (the current behavior), and only support height rendering when pixel_is_point=true.

I'm also still trying to understand what it means to provide color data in PixelIsPoint mode. Would that mean we use the colors as vertex colors and let the graphics system interpolate the colors inside the individual triangles?

[jhurliman]

I'm also still trying to understand what it means to provide color data in PixelIsPoint mode. Would that mean we use the colors as vertex colors and let the graphics system interpolate the colors inside the individual triangles?

I must have this backward, because I thought PixelIsPoint meant the value of a pixel defines a single point sample from the center of the pixel. Is that not right?

EDIT: I think I understand this as you would create a triangle mesh where the vertices fall on the center of the pixels (half-cell shift), and then there's an implementation question of how to apply the colors. If that's correct, then yes I think using vertex colors and allowing GL to interpolate makes sense.