Handle Null values in grid

[kpetrow]

I have grids with null values. Countour does not like null or undefined in the array of values. Any solutions? I can set the nulls to gridMin-1 and then set the threshold but i still Get steps down from the Larger than min values to gridMin.

[Fil]

I don't think there is a unique answer (or specification).

If null corresponds to missing data, maybe it should be ignored rather than set to the min? In that case we could to set nulls to an average of the neighbors… but then what happens when all neighbors are null too.

Also note that the current algorithm currently “works” with null (considered as 0, it creates a hole), but breaks with undefined or NaN, sometimes resulting in empty contours.

I have a test notebook here https://observablehq.com/d/c490a61e8eef537e (not very advanced).

[kpetrow]

@Fil thanks for the response. I am not able to reproduce the donuts like you are seeing. I thought at first it might be because i cant pass in "null". I am using a float32array this only allows for undefined. but i converted my float32array to a regular array and swapped out all the undefined for nulls. It gives me a rapid descent in the donuts down to the min range

                for(var i = min; i<max; i+= step){
			range.push(i);
		}
		var temp = Array.from(tempGrid.elevations);

		for(var i = 0;i<temp.length;i++){
			if( isNaN(temp[i]) )
				temp[i] = null ;
		};

		var myCountours = d3.contours()
		.size([tempGrid.metaData.nCol+1, tempGrid.metaData.nRow+1])
		.thresholds(range)
		(temp);

resulting data set. with surface model

EDIT: after rereading your post a couple times, i think you see what i am saying. Solution would be to have it skip the node completely? unknown is unknown do nothing

[Fil]

Yes—but "do nothing" is not an option, since the marching squares algorithm needs to know which way to use if it has to cross here for value=0 in this 3x1 rectangle:

-1 N/A 1

that's why I suggest doing a first pass to replace all nulls (or undefined/NaN) by neighboring values (or mean of neighboring values), if it works in this use case.

I don't think this would be in the scope of d3-contour though because 1) it's costly 2) the situation can become pretty difficult (recursive) if you have many N/A values—or even only N/A 3) most importantly, the correct solution really depends on what NaNs mean in the data model.

If NaN means for instance "sea vs land", then a value of -1 (or gridMin-1) might be the correct approach. If NaN means "missing data", then some kind of local interpolation might be better.

[Fil]

Rather than breaking silently, throw an error?

[kpetrow]

if not feasible i understand and will have to come up with a post process solution( could the lines be stacked vertically so i eliminate ones that have identical XY and are less than maxZ for xy combo) or something else. We are forming surface models from gps movements; so unknown means just that unknown.

If NaN means "missing data", then some kind of local interpolation might be better.

Sometimes its a cliff, sometimes its a gradual slope, we just cant tell what it is(its usually benches with a road).

Could you suggest an alternative method/algorithm that might work?

Rather than breaking silently, throw an error?

Yeah that might be a better solution. I still need to iterate through the flot32array to replace undefined with minRange - 1. I think throwing an error on undefined might be very helpful because it breaks the entire contour line and is confusing.

Thanks for the awesome lib.

[Fil]

If you're working with non-gridded data you might want to take a look at https://github.com/Fil/d3-tricontour

[kpetrow]

The tri-contour library does really take triangles. It takes points and Delaunay's up triangles. That wont work either because it TINs up my holes. I could make a tin by just splitting up my grid nodes, but tricontour doesn't seem to take triangles. Still searching for a solution. looks like some post process pound and ground for similar lines might be best at this point.

[kpetrow]

Just to help the next person... I decided to go with a solution of post processing. If line vertex not on the grid then toss it. and make a new segment. Bunch of little things to deal with but pretty much:

                       (
                            isNaN(tempGrid.elevations[array_pos]) ||
                            isNaN(tempGrid.elevations[array_pos - 1]) ||
                            isNaN(tempGrid.elevations[array_pos + tempGrid.metaData.nCol + 1]) ||
                            isNaN(tempGrid.elevations[array_pos - (tempGrid.metaData.nCol + 1)]) ||
                            isNaN(tempGrid.elevations[array_pos + 1])
                        ) &&
                        (
                            tempGrid.elevations[array_pos] - 1 > myCountours[i].value
                        )

Thanks for the help. Great lib

[mbostock]

We should treat null/undefined/NaN as -Infinity, i.e., a value that is lower than any possible threshold and never included in any contour.